mirror of
https://github.com/simstudioai/sim.git
synced 2026-08-30 17:05:18 +08:00
feat(tinyfish): add TinyFish web agent, search, and fetch integration (#7177)
* feat(tinyfish): add TinyFish web agent, search, and fetch integration Adds the TinyFish integration: eight tools across the Agent, Search, and Fetch APIs, a block wiring them, the brand icon, and hosted-key support. Agent runs are metered on the step count TinyFish reports. Search and Fetch are free, so their hosted key costs nothing to run. The async run and its run read/cancel/list companions carry no hosting config — their charge accrues after the request returns and cannot be metered — so they always require the caller's own key. * fix(tinyfish): address review findings - Add tinyfish to PROVIDER_SECTIONS. The sectioned BYOK renderer drops any provider missing from a section, so the key field never rendered. Export PROVIDERS/PROVIDER_SECTIONS and assert they agree, so the next provider to miss a section fails CI instead of vanishing. - Replace the `any` payload params with raw snake_case wire types. This caught two real gaps: `status` could reach the output undefined, and `error` could be null where ToolResponse.error is `string | undefined`. - Reject an already-parsed array output schema, which a `json` block input can produce, and name malformed schema JSON instead of leaking a SyntaxError. - Count Fetch rate-limit usage from the submitted URLs rather than the returned arrays, so a URL in neither array cannot undercount. - Make "List runs" a literal canvas clause; a blank goal filter lists everything. - Regenerate the docs manifest for the new integration page. * chore(byok): audit that every hosted provider is fully wired Adds check:byok-providers. A hosted tool names its provider once in hosting.byokProviderId, but the id must also reach the zod enum, the settings PROVIDERS row, and a PROVIDER_SECTIONS section. Only the union is compiler- enforced; the rest fail silently, which is how TinyFish shipped with no settings row in the first place. Also flags drift between the two BYOKProviderId declarations. Correct the Search and Fetch rate-limit rationale: TinyFish documents both ceilings per API key, not per account, so the numbered key pool does raise the total. Records why a free product is still worth hosting — the endpoints require a key, so hosting is what removes the signup.
This commit is contained in:
Waleed
File diff suppressed because one or more lines are too long
@@ -244,6 +244,7 @@ import {
|
||||
ThriveIcon,
|
||||
TikTokIcon,
|
||||
TinybirdIcon,
|
||||
TinyFishIcon,
|
||||
TrelloIcon,
|
||||
TriggerDevIcon,
|
||||
TwilioIcon,
|
||||
@@ -551,6 +552,7 @@ export const blockTypeToIconMap: Record<string, IconComponent> = {
|
||||
thrive: ThriveIcon,
|
||||
tiktok: TikTokIcon,
|
||||
tinybird: TinybirdIcon,
|
||||
tinyfish: TinyFishIcon,
|
||||
trello: TrelloIcon,
|
||||
trigger_dev: TriggerDevIcon,
|
||||
twilio: TwilioIcon,
|
||||
|
||||
@@ -258,6 +258,7 @@
|
||||
"thrive",
|
||||
"tiktok",
|
||||
"tinybird",
|
||||
"tinyfish",
|
||||
"trello",
|
||||
"trello-service-account",
|
||||
"trigger_dev",
|
||||
|
||||
@@ -0,0 +1,350 @@
|
||||
---
|
||||
title: TinyFish
|
||||
description: Automate and read the live web
|
||||
---
|
||||
|
||||
import { BlockInfoCard } from "@/components/ui/block-info-card"
|
||||
|
||||
<BlockInfoCard
|
||||
type="tinyfish"
|
||||
color="#FF6700"
|
||||
/>
|
||||
|
||||
{/* MANUAL-CONTENT-START:intro */}
|
||||
[TinyFish](https://www.tinyfish.ai/) is web infrastructure for AI agents. Instead of maintaining a scraper per site, you give a TinyFish web agent a natural-language goal and a starting URL, and it drives a real browser — clicking, typing, paginating, logging in — until the goal is met, then returns what it found as structured JSON.
|
||||
|
||||
TinyFish exposes three surfaces through this block:
|
||||
|
||||
- **Agent** — natural-language browser automation on live websites. Charged per step from a prepaid wallet.
|
||||
- **Search** — ranked web results with titles, snippets, and URLs. Free.
|
||||
- **Fetch** — up to 10 URLs at a time rendered and extracted as clean markdown, HTML, or a JSON document tree. Free.
|
||||
|
||||
With TinyFish in Sim, you can:
|
||||
|
||||
- **Automate any site, API or not**: Log into vendor portals, legacy ERPs, and internal tools that never shipped an API, and pull the data out.
|
||||
- **Get typed results, not scraped HTML**: Supply a JSON Schema in **Output Schema** and TinyFish holds the agent to it, re-prompting on mismatch and reporting every field that did not match in `schemaValidation`.
|
||||
- **Survive bot detection**: Switch **Browser Profile** to `stealth` for anti-detection, and enable the Tetra proxy with a country when the page is geo-restricted.
|
||||
- **Log in safely**: Connect a password manager to TinyFish's vault, then enable **Use Vault Credentials** and scope a run to specific credential URIs. **List Vault Items** returns those URIs as display-safe metadata — labels, domains, field names — so credentials never travel through the workflow.
|
||||
- **Run work that outlives a step**: **Start Agent Run** queues an automation and returns a run ID immediately; **Get Run**, **Cancel Run**, and **List Runs** track it afterwards, and a webhook URL can notify you on completion.
|
||||
- **Read the live web cheaply**: Pair **Search** and **Fetch URLs** to gather current sources before an agent writes or answers.
|
||||
|
||||
## Choosing an operation
|
||||
|
||||
| You want to… | Use |
|
||||
| --- | --- |
|
||||
| Get an answer back in the same workflow step | **Run Agent** |
|
||||
| Kick off a long automation and check on it later | **Start Agent Run**, then **Get Run** |
|
||||
| Stop a queued or in-flight automation | **Cancel Run** |
|
||||
| Find runs you did not record the ID for | **List Runs** |
|
||||
| Get ranked web results for a query | **Search** |
|
||||
| Read specific pages as clean text | **Fetch URLs** |
|
||||
| Find the credential URI to scope a run to | **List Vault Items** |
|
||||
|
||||
## Writing a good goal
|
||||
|
||||
The goal is handed to the agent's model verbatim, so it behaves like a prompt. Name the destination, the data, and the stopping condition — "open the pricing page and collect every plan name and monthly price" beats "get pricing". Start the run as close to the target page as you can: every navigation the URL saves is a step you are not billed for. Use **Agent Mode** `strict` when the run is a test that should fail loudly rather than improvise.
|
||||
|
||||
## API key and hosted keys
|
||||
|
||||
On Sim's hosted platform, **Run Agent**, **Search**, and **Fetch URLs** run on Sim's TinyFish key by default, metered to your workspace — Agent runs are billed per step, Search and Fetch are free. You can bring your own key in **Settings → API Keys** to bill TinyFish directly instead.
|
||||
|
||||
**Start Agent Run**, **Get Run**, **Cancel Run**, **List Runs**, and **List Vault Items** always require your own key. An async run accrues its charge after the request returns, so there is nothing for Sim to meter at call time.
|
||||
|
||||
## Errors
|
||||
|
||||
A failed automation comes back as HTTP 200 with the failure inside the run's own `error` object, so read `status` rather than assuming success. The `category` tells you what to do: `AGENT_FAILURE` means the goal or the site needs attention, `SYSTEM_FAILURE` is worth retrying after `retryAfter` seconds, and `BILLING_FAILURE` means the TinyFish wallet is empty. All of that is on the block's `error` output, so a workflow can branch on it without parsing a message string.
|
||||
{/* MANUAL-CONTENT-END */}
|
||||
|
||||
|
||||
## Usage Instructions
|
||||
|
||||
Integrate TinyFish into the workflow. Give a web agent a natural-language goal and let it drive a real browser on any site, queue and track long-running automations, search the web, and fetch pages as clean markdown.
|
||||
|
||||
|
||||
|
||||
## Actions
|
||||
|
||||
### TinyFish Run Agent
|
||||
|
||||
Run a TinyFish web agent against a website and wait for it to finish, returning the structured result it extracted
|
||||
|
||||
#### Input
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
| --------- | ---- | -------- | ----------- |
|
||||
| `url` | string | Yes | Target website URL the agent starts on |
|
||||
| `goal` | string | Yes | Natural-language description of what to accomplish on the website |
|
||||
| `browserProfile` | string | No | Browser engine: "lite" \(standard\) or "stealth" \(anti-detection\) |
|
||||
| `agentMode` | string | No | Agent behavior: "default" or "strict" \(fail fast\) |
|
||||
| `maxSteps` | number | No | Maximum tool-call steps before the agent stops \(1-500, default 150\) |
|
||||
| `outputSchema` | json | No | JSON Schema draft-07 contract the run result must satisfy |
|
||||
| `proxyEnabled` | boolean | No | Route the run through TinyFish’s Tetra proxy |
|
||||
| `proxyCountryCode` | string | No | Proxy country: US, GB, CA, DE, FR, JP, or AU |
|
||||
| `useVault` | boolean | No | Let the run use credentials from the connected TinyFish vault |
|
||||
| `credentialItemIds` | string | No | Comma-separated vault credential URIs to scope the run to |
|
||||
| `apiKey` | string | Yes | TinyFish API key |
|
||||
|
||||
#### Output
|
||||
|
||||
| Parameter | Type | Description |
|
||||
| --------- | ---- | ----------- |
|
||||
| `runId` | string | Run identifier |
|
||||
| `status` | string | Final run status: COMPLETED or FAILED |
|
||||
| `startedAt` | string | ISO 8601 timestamp when the run started |
|
||||
| `finishedAt` | string | ISO 8601 timestamp when the run finished |
|
||||
| `numOfSteps` | number | Steps the agent took |
|
||||
| `result` | json | Structured data the agent extracted, null when the run failed |
|
||||
| `schemaValidation` | object | Validation of the result against the requested output schema |
|
||||
| ↳ `valid` | boolean | Whether the result matched the requested output schema |
|
||||
| ↳ `rePromptAttempts` | number | Number of schema-repair re-prompts TinyFish performed |
|
||||
| ↳ `errors` | array | Fields that did not match the requested schema |
|
||||
| ↳ `path` | string | Path to the failing field |
|
||||
| ↳ `expected` | string | Expected type or constraint |
|
||||
| ↳ `received` | string | Type actually returned |
|
||||
| ↳ `message` | string | Validation error message |
|
||||
| `error` | object | Why the run failed, null when it succeeded. Branch on category to decide whether to retry |
|
||||
| ↳ `code` | string | Machine-readable error code |
|
||||
| ↳ `message` | string | Why the run failed |
|
||||
| ↳ `category` | string | SYSTEM_FAILURE \(retry\), AGENT_FAILURE \(fix the goal\), BILLING_FAILURE \(add credits\), or UNKNOWN |
|
||||
| ↳ `retryAfter` | number | Suggested retry delay in seconds, null when not retryable |
|
||||
| ↳ `helpUrl` | string | Troubleshooting documentation URL |
|
||||
| ↳ `helpMessage` | string | Human-readable guidance |
|
||||
|
||||
### TinyFish Start Agent Run
|
||||
|
||||
Queue a TinyFish web agent run and return its run ID immediately, without waiting for the automation to finish
|
||||
|
||||
#### Input
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
| --------- | ---- | -------- | ----------- |
|
||||
| `url` | string | Yes | Target website URL the agent starts on |
|
||||
| `goal` | string | Yes | Natural-language description of what to accomplish on the website |
|
||||
| `browserProfile` | string | No | Browser engine: "lite" \(standard\) or "stealth" \(anti-detection\) |
|
||||
| `agentMode` | string | No | Agent behavior: "default" or "strict" \(fail fast\) |
|
||||
| `maxSteps` | number | No | Maximum tool-call steps before the agent stops \(1-500, default 150\) |
|
||||
| `outputSchema` | json | No | JSON Schema draft-07 contract the run result must satisfy |
|
||||
| `proxyEnabled` | boolean | No | Route the run through TinyFish’s Tetra proxy |
|
||||
| `proxyCountryCode` | string | No | Proxy country: US, GB, CA, DE, FR, JP, or AU |
|
||||
| `useVault` | boolean | No | Let the run use credentials from the connected TinyFish vault |
|
||||
| `credentialItemIds` | string | No | Comma-separated vault credential URIs to scope the run to |
|
||||
| `apiKey` | string | Yes | TinyFish API key |
|
||||
| `webhookUrl` | string | No | HTTPS URL notified when the run completes, fails, or is cancelled |
|
||||
|
||||
#### Output
|
||||
|
||||
| Parameter | Type | Description |
|
||||
| --------- | ---- | ----------- |
|
||||
| `runId` | string | Identifier of the queued run, used to poll or cancel it |
|
||||
|
||||
### TinyFish Get Run
|
||||
|
||||
Get the status, extracted result, and step history of a TinyFish automation run by its ID
|
||||
|
||||
#### Input
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
| --------- | ---- | -------- | ----------- |
|
||||
| `runId` | string | Yes | Identifier of the run to look up |
|
||||
| `apiKey` | string | Yes | TinyFish API key |
|
||||
|
||||
#### Output
|
||||
|
||||
| Parameter | Type | Description |
|
||||
| --------- | ---- | ----------- |
|
||||
| `runId` | string | Run identifier |
|
||||
| `status` | string | PENDING, RUNNING, COMPLETED, FAILED, or CANCELLED |
|
||||
| `goal` | string | Natural-language goal the run was given |
|
||||
| `createdAt` | string | ISO 8601 timestamp when the run was created |
|
||||
| `startedAt` | string | ISO 8601 timestamp when the run started executing |
|
||||
| `finishedAt` | string | ISO 8601 timestamp when the run finished |
|
||||
| `numOfSteps` | number | Steps taken, null while the run is still in progress |
|
||||
| `result` | json | Structured data the agent extracted, null until the run succeeds |
|
||||
| `schemaValidation` | object | Validation of the result against the requested output schema |
|
||||
| ↳ `valid` | boolean | Whether the result matched the requested output schema |
|
||||
| ↳ `rePromptAttempts` | number | Number of schema-repair re-prompts TinyFish performed |
|
||||
| ↳ `errors` | array | Fields that did not match the requested schema |
|
||||
| ↳ `path` | string | Path to the failing field |
|
||||
| ↳ `expected` | string | Expected type or constraint |
|
||||
| ↳ `received` | string | Type actually returned |
|
||||
| ↳ `message` | string | Validation error message |
|
||||
| `error` | object | Failure details, null while the run is pending or succeeded |
|
||||
| ↳ `code` | string | Machine-readable error code |
|
||||
| ↳ `message` | string | Why the run failed |
|
||||
| ↳ `category` | string | SYSTEM_FAILURE \(retry\), AGENT_FAILURE \(fix the goal\), BILLING_FAILURE \(add credits\), or UNKNOWN |
|
||||
| ↳ `retryAfter` | number | Suggested retry delay in seconds, null when not retryable |
|
||||
| ↳ `helpUrl` | string | Troubleshooting documentation URL |
|
||||
| ↳ `helpMessage` | string | Human-readable guidance |
|
||||
| `streamingUrl` | string | Live browser view URL, available while the run is executing |
|
||||
| `browserConfig` | object | Proxy settings the run executed with |
|
||||
| ↳ `proxyEnabled` | boolean | Whether a proxy was used |
|
||||
| ↳ `proxyCountryCode` | string | Proxy country code |
|
||||
| `videoUrl` | string | Presigned recording URL, expires 15 minutes after it is issued |
|
||||
| `steps` | array | Steps the agent took during the run |
|
||||
| ↳ `id` | string | Step identifier |
|
||||
| ↳ `timestamp` | string | ISO 8601 timestamp of the step |
|
||||
| ↳ `status` | string | Status of the run at this step |
|
||||
| ↳ `action` | string | Action the agent took |
|
||||
| ↳ `duration` | string | Time the step took |
|
||||
|
||||
### TinyFish Cancel Run
|
||||
|
||||
Cancel a queued or in-progress TinyFish automation run by its ID
|
||||
|
||||
#### Input
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
| --------- | ---- | -------- | ----------- |
|
||||
| `runId` | string | Yes | Identifier of the run to cancel |
|
||||
| `apiKey` | string | Yes | TinyFish API key |
|
||||
|
||||
#### Output
|
||||
|
||||
| Parameter | Type | Description |
|
||||
| --------- | ---- | ----------- |
|
||||
| `runId` | string | Run identifier |
|
||||
| `status` | string | Status after the call: CANCELLED, or the terminal status the run already reached |
|
||||
| `cancelledAt` | string | ISO 8601 timestamp of the cancellation, null when nothing was cancelled |
|
||||
| `message` | string | Context such as "Run already cancelled" or "Run already finished" |
|
||||
|
||||
### TinyFish List Runs
|
||||
|
||||
List TinyFish automation runs, optionally filtered by status, goal text, or creation date
|
||||
|
||||
#### Input
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
| --------- | ---- | -------- | ----------- |
|
||||
| `status` | string | No | Filter by run status: PENDING, RUNNING, COMPLETED, FAILED, or CANCELLED |
|
||||
| `goal` | string | No | Filter by goal text \(case-insensitive partial match, max 500 characters\) |
|
||||
| `createdAfter` | string | No | Only return runs created after this ISO 8601 timestamp |
|
||||
| `createdBefore` | string | No | Only return runs created before this ISO 8601 timestamp |
|
||||
| `sortDirection` | string | No | Sort by creation time: "desc" \(newest first, default\) or "asc" |
|
||||
| `limit` | number | No | Maximum runs to return \(1-100, default 20\) |
|
||||
| `cursor` | string | No | Pagination cursor returned by a previous call |
|
||||
| `apiKey` | string | Yes | TinyFish API key |
|
||||
|
||||
#### Output
|
||||
|
||||
| Parameter | Type | Description |
|
||||
| --------- | ---- | ----------- |
|
||||
| `runs` | array | Runs matching the filters, newest first by default |
|
||||
| ↳ `runId` | string | Run identifier |
|
||||
| ↳ `status` | string | PENDING, RUNNING, COMPLETED, FAILED, or CANCELLED |
|
||||
| ↳ `goal` | string | Natural-language goal the run was given |
|
||||
| ↳ `createdAt` | string | ISO 8601 timestamp when the run was created |
|
||||
| ↳ `startedAt` | string | ISO 8601 timestamp when the run started executing |
|
||||
| ↳ `finishedAt` | string | ISO 8601 timestamp when the run finished |
|
||||
| ↳ `numOfSteps` | number | Steps taken, null while the run is still in progress |
|
||||
| ↳ `result` | json | Structured data the agent extracted, null until the run succeeds |
|
||||
| ↳ `schemaValidation` | object | Validation of the result against the requested output schema |
|
||||
| ↳ `valid` | boolean | Whether the result matched the requested output schema |
|
||||
| ↳ `rePromptAttempts` | number | Number of schema-repair re-prompts TinyFish performed |
|
||||
| ↳ `errors` | array | Fields that did not match the requested schema |
|
||||
| ↳ `path` | string | Path to the failing field |
|
||||
| ↳ `expected` | string | Expected type or constraint |
|
||||
| ↳ `received` | string | Type actually returned |
|
||||
| ↳ `message` | string | Validation error message |
|
||||
| ↳ `error` | object | Failure details, null while the run is pending or succeeded |
|
||||
| ↳ `code` | string | Machine-readable error code |
|
||||
| ↳ `message` | string | Why the run failed |
|
||||
| ↳ `category` | string | SYSTEM_FAILURE \(retry\), AGENT_FAILURE \(fix the goal\), BILLING_FAILURE \(add credits\), or UNKNOWN |
|
||||
| ↳ `retryAfter` | number | Suggested retry delay in seconds, null when not retryable |
|
||||
| ↳ `helpUrl` | string | Troubleshooting documentation URL |
|
||||
| ↳ `helpMessage` | string | Human-readable guidance |
|
||||
| ↳ `streamingUrl` | string | Live browser view URL, available while the run is executing |
|
||||
| ↳ `browserConfig` | object | Proxy settings the run executed with |
|
||||
| ↳ `proxyEnabled` | boolean | Whether a proxy was used |
|
||||
| ↳ `proxyCountryCode` | string | Proxy country code |
|
||||
| `total` | number | Total runs matching the filters |
|
||||
| `nextCursor` | string | Cursor for the next page, null when there are no more results |
|
||||
| `hasMore` | boolean | Whether more results follow this page |
|
||||
|
||||
### TinyFish Search
|
||||
|
||||
Search the web with TinyFish and get ranked results with titles, snippets, and URLs
|
||||
|
||||
#### Input
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
| --------- | ---- | -------- | ----------- |
|
||||
| `query` | string | Yes | Search query |
|
||||
| `location` | string | No | Country code for geo-targeted results, such as US |
|
||||
| `language` | string | No | Language code for the results, such as en |
|
||||
| `apiKey` | string | Yes | TinyFish API key |
|
||||
|
||||
#### Output
|
||||
|
||||
| Parameter | Type | Description |
|
||||
| --------- | ---- | ----------- |
|
||||
| `query` | string | Query that was executed |
|
||||
| `results` | array | Ranked search results |
|
||||
| ↳ `position` | number | Rank in the result list |
|
||||
| ↳ `siteName` | string | Site the result came from |
|
||||
| ↳ `snippet` | string | Text snippet from the page |
|
||||
| ↳ `title` | string | Page title |
|
||||
| ↳ `url` | string | Result URL |
|
||||
| `totalResults` | number | Number of results returned |
|
||||
|
||||
### TinyFish Fetch
|
||||
|
||||
Fetch up to 10 URLs with TinyFish, rendering JavaScript when needed, and return clean extracted content
|
||||
|
||||
#### Input
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
| --------- | ---- | -------- | ----------- |
|
||||
| `urls` | string | Yes | Comma-separated list of 1-10 URLs to fetch |
|
||||
| `format` | string | No | Extraction format: "markdown" \(default\), "html", or "json" |
|
||||
| `links` | boolean | No | Also return every outbound link found on each page |
|
||||
| `imageLinks` | boolean | No | Also return every image URL found on each page |
|
||||
| `apiKey` | string | Yes | TinyFish API key |
|
||||
|
||||
#### Output
|
||||
|
||||
| Parameter | Type | Description |
|
||||
| --------- | ---- | ----------- |
|
||||
| `results` | array | Successfully fetched pages |
|
||||
| ↳ `url` | string | URL that was requested |
|
||||
| ↳ `finalUrl` | string | URL after redirects |
|
||||
| ↳ `title` | string | Page title |
|
||||
| ↳ `description` | string | Meta description |
|
||||
| ↳ `language` | string | Detected language code |
|
||||
| ↳ `format` | string | Format of the extracted content |
|
||||
| ↳ `text` | json | Extracted content — a string for markdown and html, a document tree for json |
|
||||
| ↳ `author` | string | Page author |
|
||||
| ↳ `publishedDate` | string | Published date |
|
||||
| ↳ `links` | array | Outbound links, only when links was requested |
|
||||
| ↳ `imageLinks` | array | Image URLs, only when image links were requested |
|
||||
| ↳ `latencyMs` | number | Fetch latency in ms |
|
||||
| `errors` | array | URLs that failed. A per-URL failure never fails the whole request |
|
||||
| ↳ `url` | string | URL that failed |
|
||||
| ↳ `error` | string | Why the fetch failed |
|
||||
|
||||
### TinyFish List Vault Items
|
||||
|
||||
List the credentials available from password managers connected to TinyFish, with the URIs an agent run can be scoped to
|
||||
|
||||
#### Input
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
| --------- | ---- | -------- | ----------- |
|
||||
| `apiKey` | string | Yes | TinyFish API key |
|
||||
|
||||
#### Output
|
||||
|
||||
| Parameter | Type | Description |
|
||||
| --------- | ---- | ----------- |
|
||||
| `items` | array | Credentials available to automation runs |
|
||||
| ↳ `itemId` | string | Credential URI, used as a Vault Credential URI on a run |
|
||||
| ↳ `connectionId` | string | Identifier of the vault connection it came from |
|
||||
| ↳ `label` | string | Credential name, such as "Amazon Login" |
|
||||
| ↳ `vaultName` | string | Vault the credential lives in |
|
||||
| ↳ `domains` | array | Domains the credential applies to |
|
||||
| ↳ `fieldMetadata` | array | Fields the credential carries, without their values |
|
||||
| ↳ `fieldId` | string | Field identifier |
|
||||
| ↳ `label` | string | Field name |
|
||||
| ↳ `type` | string | STRING, CONCEALED, or OTP |
|
||||
| ↳ `hasTotp` | boolean | Whether the credential carries a TOTP secret |
|
||||
|
||||
|
||||
@@ -36,6 +36,7 @@ import {
|
||||
PerplexityIcon,
|
||||
ProspeoIcon,
|
||||
SerperIcon,
|
||||
TinyFishIcon,
|
||||
TogetherIcon,
|
||||
WizaIcon,
|
||||
xAIIcon,
|
||||
@@ -183,6 +184,13 @@ const PROVIDERS: (BYOKManagerProvider & { id: BYOKProviderId })[] = [
|
||||
description: 'Web scraping, crawling, search, and brand intelligence',
|
||||
placeholder: 'Enter your Context.dev API key',
|
||||
},
|
||||
{
|
||||
id: 'tinyfish',
|
||||
name: 'TinyFish',
|
||||
icon: TinyFishIcon,
|
||||
description: 'Web agent automation, search, and page fetching',
|
||||
placeholder: 'Enter your TinyFish API key',
|
||||
},
|
||||
{
|
||||
id: 'serper',
|
||||
name: 'Serper',
|
||||
@@ -355,6 +363,7 @@ const PROVIDER_SECTIONS: BYOKProviderSection[] = [
|
||||
'firecrawl',
|
||||
'exa',
|
||||
'context_dev',
|
||||
'tinyfish',
|
||||
'serper',
|
||||
'linkup',
|
||||
'parallel_ai',
|
||||
|
||||
@@ -0,0 +1,696 @@
|
||||
import { TinyFishIcon } from '@/components/icons'
|
||||
import type { BlockConfig, BlockMeta } from '@/blocks/types'
|
||||
import { AuthMode, IntegrationType } from '@/blocks/types'
|
||||
import type { TinyFishRunResponse } from '@/tools/tinyfish/types'
|
||||
|
||||
/** Operations that build and submit an automation run. */
|
||||
const AUTOMATION_OPERATIONS = ['tinyfish_run', 'tinyfish_run_async']
|
||||
|
||||
/** Operations addressed by a single run ID. */
|
||||
const RUN_ID_OPERATIONS = ['tinyfish_get_run', 'tinyfish_cancel_run']
|
||||
|
||||
/**
|
||||
* Operations whose tools declare a `hosting` config.
|
||||
*
|
||||
* The async run bills for steps taken after the request returns, and the run
|
||||
* read/cancel/list endpoints are companions to it, so none of them can be
|
||||
* metered against a hosted key. They always require the caller's own key.
|
||||
*/
|
||||
const HOSTED_KEY_OPERATIONS = ['tinyfish_run', 'tinyfish_search', 'tinyfish_fetch']
|
||||
|
||||
const PROXY_COUNTRY_OPTIONS = [
|
||||
{ label: 'United States', id: 'US' },
|
||||
{ label: 'United Kingdom', id: 'GB' },
|
||||
{ label: 'Canada', id: 'CA' },
|
||||
{ label: 'Germany', id: 'DE' },
|
||||
{ label: 'France', id: 'FR' },
|
||||
{ label: 'Japan', id: 'JP' },
|
||||
{ label: 'Australia', id: 'AU' },
|
||||
]
|
||||
|
||||
export const TinyFishBlock: BlockConfig<TinyFishRunResponse> = {
|
||||
type: 'tinyfish',
|
||||
name: 'TinyFish',
|
||||
description: 'Automate and read the live web',
|
||||
authMode: AuthMode.ApiKey,
|
||||
longDescription:
|
||||
'Integrate TinyFish into the workflow. Give a web agent a natural-language goal and let it drive a real browser on any site, queue and track long-running automations, search the web, and fetch pages as clean markdown.',
|
||||
docsLink: 'https://docs.sim.ai/integrations/tinyfish',
|
||||
category: 'tools',
|
||||
integrationType: IntegrationType.AI,
|
||||
bgColor: '#FF6700',
|
||||
icon: TinyFishIcon,
|
||||
canvasPresentation: {
|
||||
defaultTitle: 'TinyFish',
|
||||
sentences: {
|
||||
byOperation: {
|
||||
tinyfish_run: [
|
||||
{ text: 'Run the goal', field: 'goal', core: true },
|
||||
{ text: 'on', field: 'url' },
|
||||
],
|
||||
tinyfish_run_async: [
|
||||
{ text: 'Queue the goal', field: 'goal', core: true },
|
||||
{ text: 'on', field: 'url' },
|
||||
],
|
||||
tinyfish_get_run: [{ text: 'Get run', field: 'runId', core: true }],
|
||||
tinyfish_cancel_run: [{ text: 'Cancel run', field: 'runId', core: true }],
|
||||
tinyfish_list_runs: [
|
||||
'List runs',
|
||||
{ text: 'matching', field: 'goalFilter' },
|
||||
{ text: ', with status', field: 'status' },
|
||||
],
|
||||
tinyfish_search: [{ text: 'Search the web for', field: 'query', core: true }],
|
||||
tinyfish_fetch: [
|
||||
{ text: 'Fetch', field: 'urls', core: true },
|
||||
{ text: 'as', field: 'format' },
|
||||
],
|
||||
tinyfish_list_vault_items: ['List credentials from the connected vault'],
|
||||
},
|
||||
},
|
||||
},
|
||||
subBlocks: [
|
||||
{
|
||||
id: 'operation',
|
||||
title: 'Operation',
|
||||
type: 'dropdown',
|
||||
options: [
|
||||
{ label: 'Run Agent', id: 'tinyfish_run' },
|
||||
{ label: 'Start Agent Run', id: 'tinyfish_run_async' },
|
||||
{ label: 'Get Run', id: 'tinyfish_get_run' },
|
||||
{ label: 'Cancel Run', id: 'tinyfish_cancel_run' },
|
||||
{ label: 'List Runs', id: 'tinyfish_list_runs' },
|
||||
{ label: 'Search', id: 'tinyfish_search' },
|
||||
{ label: 'Fetch URLs', id: 'tinyfish_fetch' },
|
||||
{ label: 'List Vault Items', id: 'tinyfish_list_vault_items' },
|
||||
],
|
||||
value: () => 'tinyfish_run',
|
||||
},
|
||||
|
||||
{
|
||||
id: 'url',
|
||||
title: 'Website URL',
|
||||
type: 'short-input',
|
||||
placeholder: 'https://example.com',
|
||||
condition: { field: 'operation', value: AUTOMATION_OPERATIONS },
|
||||
required: { field: 'operation', value: AUTOMATION_OPERATIONS },
|
||||
},
|
||||
{
|
||||
id: 'goal',
|
||||
title: 'Goal',
|
||||
type: 'long-input',
|
||||
placeholder: 'Find the pricing page and extract every plan name and monthly price',
|
||||
condition: { field: 'operation', value: AUTOMATION_OPERATIONS },
|
||||
required: { field: 'operation', value: AUTOMATION_OPERATIONS },
|
||||
},
|
||||
{
|
||||
id: 'outputSchema',
|
||||
title: 'Output Schema',
|
||||
type: 'code',
|
||||
language: 'json',
|
||||
placeholder: '{\n "type": "object",\n "properties": {}\n}',
|
||||
description: 'JSON Schema draft-07 contract the extracted result must satisfy',
|
||||
condition: { field: 'operation', value: AUTOMATION_OPERATIONS },
|
||||
mode: 'advanced',
|
||||
wandConfig: {
|
||||
enabled: true,
|
||||
prompt: `Generate a JSON Schema (draft-07) describing the data the web agent should return.
|
||||
|
||||
Rules:
|
||||
- Root must be an object with a "type" and "properties".
|
||||
- Use "array" with "items" for repeated records such as products, plans, or listings.
|
||||
- Only include fields the agent could actually read off the page.
|
||||
- Mark the fields that must always be present in "required".
|
||||
|
||||
Example:
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"plans": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"name": { "type": "string" },
|
||||
"monthlyPrice": { "type": "number" }
|
||||
},
|
||||
"required": ["name", "monthlyPrice"]
|
||||
}
|
||||
}
|
||||
},
|
||||
"required": ["plans"]
|
||||
}
|
||||
|
||||
Return ONLY the JSON Schema - no explanations, no extra text.`,
|
||||
placeholder: 'Describe the data you want the agent to bring back...',
|
||||
generationType: 'json-schema',
|
||||
},
|
||||
},
|
||||
{
|
||||
id: 'browserProfile',
|
||||
title: 'Browser Profile',
|
||||
type: 'dropdown',
|
||||
options: [
|
||||
{ label: 'Lite (standard browser)', id: 'lite' },
|
||||
{ label: 'Stealth (anti-detection)', id: 'stealth' },
|
||||
],
|
||||
value: () => 'lite',
|
||||
condition: { field: 'operation', value: AUTOMATION_OPERATIONS },
|
||||
mode: 'advanced',
|
||||
},
|
||||
{
|
||||
id: 'agentMode',
|
||||
title: 'Agent Mode',
|
||||
type: 'dropdown',
|
||||
options: [
|
||||
{ label: 'Default', id: 'default' },
|
||||
{ label: 'Strict (fail fast)', id: 'strict' },
|
||||
],
|
||||
value: () => 'default',
|
||||
condition: { field: 'operation', value: AUTOMATION_OPERATIONS },
|
||||
mode: 'advanced',
|
||||
},
|
||||
{
|
||||
id: 'maxSteps',
|
||||
title: 'Max Steps',
|
||||
type: 'short-input',
|
||||
placeholder: '150',
|
||||
description: 'Tool-call steps before the agent stops (1-500)',
|
||||
condition: { field: 'operation', value: AUTOMATION_OPERATIONS },
|
||||
mode: 'advanced',
|
||||
},
|
||||
{
|
||||
id: 'proxyEnabled',
|
||||
title: 'Use Proxy',
|
||||
type: 'switch',
|
||||
description: 'Route the run through TinyFish’s Tetra proxy',
|
||||
condition: { field: 'operation', value: AUTOMATION_OPERATIONS },
|
||||
mode: 'advanced',
|
||||
},
|
||||
{
|
||||
id: 'proxyCountryCode',
|
||||
title: 'Proxy Country',
|
||||
type: 'dropdown',
|
||||
options: PROXY_COUNTRY_OPTIONS,
|
||||
condition: {
|
||||
field: 'operation',
|
||||
value: AUTOMATION_OPERATIONS,
|
||||
and: { field: 'proxyEnabled', value: true },
|
||||
},
|
||||
mode: 'advanced',
|
||||
},
|
||||
{
|
||||
id: 'useVault',
|
||||
title: 'Use Vault Credentials',
|
||||
type: 'switch',
|
||||
description: 'Let the agent log in with credentials from your connected password manager',
|
||||
condition: { field: 'operation', value: AUTOMATION_OPERATIONS },
|
||||
mode: 'advanced',
|
||||
},
|
||||
{
|
||||
id: 'credentialItemIds',
|
||||
title: 'Vault Credential URIs',
|
||||
type: 'short-input',
|
||||
placeholder: 'cred:conn-abc:Work:item-123, cred:conn-def:Personal:item-456',
|
||||
description:
|
||||
'Comma-separated. Run List Vault Items to find them. Leave empty to use every enabled vault item',
|
||||
condition: {
|
||||
field: 'operation',
|
||||
value: AUTOMATION_OPERATIONS,
|
||||
and: { field: 'useVault', value: true },
|
||||
},
|
||||
mode: 'advanced',
|
||||
},
|
||||
{
|
||||
id: 'webhookUrl',
|
||||
title: 'Webhook URL',
|
||||
type: 'short-input',
|
||||
placeholder: 'https://example.com/tinyfish-webhook',
|
||||
description: 'HTTPS endpoint notified when the run completes, fails, or is cancelled',
|
||||
condition: { field: 'operation', value: 'tinyfish_run_async' },
|
||||
mode: 'advanced',
|
||||
},
|
||||
|
||||
{
|
||||
id: 'runId',
|
||||
title: 'Run ID',
|
||||
type: 'short-input',
|
||||
placeholder: 'a1b2c3d4-e5f6-7890-abcd-ef1234567890',
|
||||
condition: { field: 'operation', value: RUN_ID_OPERATIONS },
|
||||
required: { field: 'operation', value: RUN_ID_OPERATIONS },
|
||||
},
|
||||
|
||||
{
|
||||
id: 'status',
|
||||
title: 'Status',
|
||||
type: 'dropdown',
|
||||
options: [
|
||||
{ label: 'Any', id: '' },
|
||||
{ label: 'Pending', id: 'PENDING' },
|
||||
{ label: 'Running', id: 'RUNNING' },
|
||||
{ label: 'Completed', id: 'COMPLETED' },
|
||||
{ label: 'Failed', id: 'FAILED' },
|
||||
{ label: 'Cancelled', id: 'CANCELLED' },
|
||||
],
|
||||
value: () => '',
|
||||
condition: { field: 'operation', value: 'tinyfish_list_runs' },
|
||||
},
|
||||
{
|
||||
id: 'goalFilter',
|
||||
title: 'Goal Contains',
|
||||
canvasNoun: 'a goal',
|
||||
type: 'short-input',
|
||||
placeholder: 'pricing',
|
||||
condition: { field: 'operation', value: 'tinyfish_list_runs' },
|
||||
},
|
||||
{
|
||||
id: 'limit',
|
||||
title: 'Limit',
|
||||
type: 'short-input',
|
||||
placeholder: '20',
|
||||
description: 'Runs to return (1-100)',
|
||||
condition: { field: 'operation', value: 'tinyfish_list_runs' },
|
||||
mode: 'advanced',
|
||||
},
|
||||
{
|
||||
id: 'createdAfter',
|
||||
title: 'Created After',
|
||||
type: 'short-input',
|
||||
placeholder: '2026-01-01T00:00:00Z',
|
||||
condition: { field: 'operation', value: 'tinyfish_list_runs' },
|
||||
mode: 'advanced',
|
||||
wandConfig: {
|
||||
enabled: true,
|
||||
prompt: `Generate an ISO 8601 timestamp based on the user's description.
|
||||
|
||||
Examples:
|
||||
- "yesterday at midnight UTC" -> 2026-01-01T00:00:00Z
|
||||
- "start of last week" -> appropriate ISO 8601 date
|
||||
- "3 days ago" -> appropriate ISO 8601 date
|
||||
|
||||
Return ONLY the ISO 8601 timestamp - no explanations, no quotes, no extra text.`,
|
||||
placeholder: 'Describe the cutoff (e.g., "3 days ago")...',
|
||||
generationType: 'timestamp',
|
||||
},
|
||||
},
|
||||
{
|
||||
id: 'createdBefore',
|
||||
title: 'Created Before',
|
||||
type: 'short-input',
|
||||
placeholder: '2026-02-01T00:00:00Z',
|
||||
condition: { field: 'operation', value: 'tinyfish_list_runs' },
|
||||
mode: 'advanced',
|
||||
wandConfig: {
|
||||
enabled: true,
|
||||
prompt: `Generate an ISO 8601 timestamp based on the user's description.
|
||||
|
||||
Examples:
|
||||
- "yesterday at midnight UTC" -> 2026-01-01T00:00:00Z
|
||||
- "start of last week" -> appropriate ISO 8601 date
|
||||
- "3 days ago" -> appropriate ISO 8601 date
|
||||
|
||||
Return ONLY the ISO 8601 timestamp - no explanations, no quotes, no extra text.`,
|
||||
placeholder: 'Describe the cutoff (e.g., "3 days ago")...',
|
||||
generationType: 'timestamp',
|
||||
},
|
||||
},
|
||||
{
|
||||
id: 'sortDirection',
|
||||
title: 'Sort Direction',
|
||||
type: 'dropdown',
|
||||
options: [
|
||||
{ label: 'Newest first', id: 'desc' },
|
||||
{ label: 'Oldest first', id: 'asc' },
|
||||
],
|
||||
value: () => 'desc',
|
||||
condition: { field: 'operation', value: 'tinyfish_list_runs' },
|
||||
mode: 'advanced',
|
||||
},
|
||||
{
|
||||
id: 'cursor',
|
||||
title: 'Cursor',
|
||||
type: 'short-input',
|
||||
placeholder: 'Cursor from a previous page',
|
||||
condition: { field: 'operation', value: 'tinyfish_list_runs' },
|
||||
mode: 'advanced',
|
||||
},
|
||||
|
||||
{
|
||||
id: 'query',
|
||||
title: 'Query',
|
||||
type: 'long-input',
|
||||
placeholder: 'web automation tools',
|
||||
condition: { field: 'operation', value: 'tinyfish_search' },
|
||||
required: { field: 'operation', value: 'tinyfish_search' },
|
||||
},
|
||||
{
|
||||
id: 'location',
|
||||
title: 'Location',
|
||||
type: 'short-input',
|
||||
placeholder: 'US',
|
||||
description: 'Country code for geo-targeted results',
|
||||
condition: { field: 'operation', value: 'tinyfish_search' },
|
||||
mode: 'advanced',
|
||||
},
|
||||
{
|
||||
id: 'language',
|
||||
title: 'Language',
|
||||
type: 'short-input',
|
||||
placeholder: 'en',
|
||||
description: 'Language code for the results',
|
||||
condition: { field: 'operation', value: 'tinyfish_search' },
|
||||
mode: 'advanced',
|
||||
},
|
||||
|
||||
{
|
||||
id: 'urls',
|
||||
title: 'URLs',
|
||||
type: 'long-input',
|
||||
placeholder: 'https://example.com, https://example.org',
|
||||
description: 'Comma-separated, 1-10 URLs. Each is fetched independently',
|
||||
condition: { field: 'operation', value: 'tinyfish_fetch' },
|
||||
required: { field: 'operation', value: 'tinyfish_fetch' },
|
||||
wandConfig: {
|
||||
enabled: true,
|
||||
prompt: `Generate a comma-separated list of at most 10 absolute URLs from the user's description.
|
||||
|
||||
Rules:
|
||||
- Every entry must start with http:// or https://.
|
||||
- Separate entries with a comma and a space.
|
||||
- Never invent a page that would not plausibly exist.
|
||||
|
||||
Return ONLY the comma-separated URL list - no explanations, no extra text.`,
|
||||
placeholder: 'Describe the pages to fetch...',
|
||||
},
|
||||
},
|
||||
{
|
||||
id: 'format',
|
||||
title: 'Format',
|
||||
type: 'dropdown',
|
||||
options: [
|
||||
{ label: 'Markdown', id: 'markdown' },
|
||||
{ label: 'HTML', id: 'html' },
|
||||
{ label: 'JSON document tree', id: 'json' },
|
||||
],
|
||||
value: () => 'markdown',
|
||||
condition: { field: 'operation', value: 'tinyfish_fetch' },
|
||||
},
|
||||
{
|
||||
id: 'links',
|
||||
title: 'Extract Links',
|
||||
type: 'switch',
|
||||
condition: { field: 'operation', value: 'tinyfish_fetch' },
|
||||
mode: 'advanced',
|
||||
},
|
||||
{
|
||||
id: 'imageLinks',
|
||||
title: 'Extract Image Links',
|
||||
type: 'switch',
|
||||
condition: { field: 'operation', value: 'tinyfish_fetch' },
|
||||
mode: 'advanced',
|
||||
},
|
||||
|
||||
{
|
||||
id: 'apiKey',
|
||||
title: 'API Key',
|
||||
type: 'short-input',
|
||||
placeholder: 'Enter your TinyFish API key',
|
||||
password: true,
|
||||
required: true,
|
||||
hideWhenHosted: true,
|
||||
condition: { field: 'operation', value: HOSTED_KEY_OPERATIONS },
|
||||
},
|
||||
{
|
||||
id: 'apiKey',
|
||||
title: 'API Key',
|
||||
type: 'short-input',
|
||||
placeholder: 'Enter your TinyFish API key',
|
||||
password: true,
|
||||
required: true,
|
||||
condition: { field: 'operation', value: HOSTED_KEY_OPERATIONS, not: true },
|
||||
},
|
||||
],
|
||||
tools: {
|
||||
access: [
|
||||
'tinyfish_run',
|
||||
'tinyfish_run_async',
|
||||
'tinyfish_get_run',
|
||||
'tinyfish_cancel_run',
|
||||
'tinyfish_list_runs',
|
||||
'tinyfish_search',
|
||||
'tinyfish_fetch',
|
||||
'tinyfish_list_vault_items',
|
||||
],
|
||||
config: {
|
||||
tool: (params) => {
|
||||
switch (params.operation) {
|
||||
case 'tinyfish_run_async':
|
||||
return 'tinyfish_run_async'
|
||||
case 'tinyfish_get_run':
|
||||
return 'tinyfish_get_run'
|
||||
case 'tinyfish_cancel_run':
|
||||
return 'tinyfish_cancel_run'
|
||||
case 'tinyfish_list_runs':
|
||||
return 'tinyfish_list_runs'
|
||||
case 'tinyfish_search':
|
||||
return 'tinyfish_search'
|
||||
case 'tinyfish_fetch':
|
||||
return 'tinyfish_fetch'
|
||||
case 'tinyfish_list_vault_items':
|
||||
return 'tinyfish_list_vault_items'
|
||||
default:
|
||||
return 'tinyfish_run'
|
||||
}
|
||||
},
|
||||
params: (params) => {
|
||||
const result: Record<string, unknown> = {}
|
||||
|
||||
const maxSteps = String(params.maxSteps ?? '').trim()
|
||||
if (maxSteps) result.maxSteps = Number(maxSteps)
|
||||
|
||||
const limit = String(params.limit ?? '').trim()
|
||||
if (limit) result.limit = Number(limit)
|
||||
|
||||
/**
|
||||
* The list filter has its own sub-block id so it does not collide with
|
||||
* the automation goal, and is renamed here to the `goal` query the tool
|
||||
* sends.
|
||||
*/
|
||||
if (params.operation === 'tinyfish_list_runs') {
|
||||
result.goal = params.goalFilter
|
||||
}
|
||||
|
||||
return result
|
||||
},
|
||||
},
|
||||
},
|
||||
inputs: {
|
||||
operation: { type: 'string', description: 'Operation to perform' },
|
||||
apiKey: { type: 'string', description: 'TinyFish API key' },
|
||||
url: { type: 'string', description: 'Website the agent starts on' },
|
||||
goal: { type: 'string', description: 'Natural-language goal for the agent' },
|
||||
outputSchema: { type: 'json', description: 'JSON Schema contract for the result' },
|
||||
browserProfile: { type: 'string', description: 'Browser engine: lite or stealth' },
|
||||
agentMode: { type: 'string', description: 'Agent behavior: default or strict' },
|
||||
maxSteps: { type: 'number', description: 'Maximum agent steps' },
|
||||
proxyEnabled: { type: 'boolean', description: 'Route the run through the Tetra proxy' },
|
||||
proxyCountryCode: { type: 'string', description: 'Proxy country code' },
|
||||
useVault: { type: 'boolean', description: 'Allow vault credentials during the run' },
|
||||
credentialItemIds: { type: 'string', description: 'Comma-separated vault credential URIs' },
|
||||
webhookUrl: { type: 'string', description: 'HTTPS endpoint notified on run lifecycle events' },
|
||||
runId: { type: 'string', description: 'Run identifier' },
|
||||
status: { type: 'string', description: 'Run status filter' },
|
||||
goalFilter: { type: 'string', description: 'Goal text filter' },
|
||||
createdAfter: { type: 'string', description: 'Only runs created after this timestamp' },
|
||||
createdBefore: { type: 'string', description: 'Only runs created before this timestamp' },
|
||||
sortDirection: { type: 'string', description: 'Sort order by creation time' },
|
||||
cursor: { type: 'string', description: 'Pagination cursor' },
|
||||
limit: { type: 'number', description: 'Maximum runs to return' },
|
||||
query: { type: 'string', description: 'Search query' },
|
||||
location: { type: 'string', description: 'Country code for geo-targeted results' },
|
||||
language: { type: 'string', description: 'Language code for the results' },
|
||||
urls: { type: 'string', description: 'Comma-separated URLs to fetch' },
|
||||
format: { type: 'string', description: 'Fetch extraction format' },
|
||||
links: { type: 'boolean', description: 'Extract outbound links' },
|
||||
imageLinks: { type: 'boolean', description: 'Extract image links' },
|
||||
},
|
||||
outputs: {
|
||||
runId: { type: 'string', description: 'Run identifier' },
|
||||
status: { type: 'string', description: 'Run status' },
|
||||
goal: { type: 'string', description: 'Goal the run was given' },
|
||||
createdAt: { type: 'string', description: 'When the run was created' },
|
||||
startedAt: { type: 'string', description: 'When the run started executing' },
|
||||
finishedAt: { type: 'string', description: 'When the run finished' },
|
||||
cancelledAt: { type: 'string', description: 'When the run was cancelled' },
|
||||
numOfSteps: { type: 'number', description: 'Steps the agent took' },
|
||||
result: { type: 'json', description: 'Structured data the agent extracted' },
|
||||
schemaValidation: {
|
||||
type: 'json',
|
||||
description:
|
||||
'Result validation against the output schema (valid, rePromptAttempts, errors[{path, expected, received, message}])',
|
||||
},
|
||||
error: {
|
||||
type: 'json',
|
||||
description:
|
||||
'Failure details for a failed run (code, message, category, retryAfter, helpUrl, helpMessage)',
|
||||
},
|
||||
streamingUrl: { type: 'string', description: 'Live browser view URL while the run executes' },
|
||||
videoUrl: { type: 'string', description: 'Presigned recording URL, expires in 15 minutes' },
|
||||
browserConfig: {
|
||||
type: 'json',
|
||||
description: 'Proxy settings the run executed with (proxyEnabled, proxyCountryCode)',
|
||||
},
|
||||
steps: {
|
||||
type: 'json',
|
||||
description:
|
||||
'Steps the agent took during the run [{id, timestamp, status, action, duration}]',
|
||||
},
|
||||
message: { type: 'string', description: 'Context returned by a cancellation' },
|
||||
runs: {
|
||||
type: 'json',
|
||||
description:
|
||||
'Runs matching the list filters [{runId, status, goal, createdAt, startedAt, finishedAt, numOfSteps, result, schemaValidation, error, streamingUrl, browserConfig}]',
|
||||
},
|
||||
total: { type: 'number', description: 'Total runs matching the list filters' },
|
||||
nextCursor: { type: 'string', description: 'Cursor for the next page of runs' },
|
||||
hasMore: { type: 'boolean', description: 'Whether more runs follow this page' },
|
||||
query: { type: 'string', description: 'Search query that was executed' },
|
||||
results: {
|
||||
type: 'json',
|
||||
description:
|
||||
'Search results [{position, siteName, snippet, title, url}], or fetched pages [{url, finalUrl, title, description, language, format, text, author, publishedDate, links, imageLinks, latencyMs}]',
|
||||
},
|
||||
totalResults: { type: 'number', description: 'Number of search results returned' },
|
||||
errors: { type: 'json', description: 'URLs the fetch could not retrieve [{url, error}]' },
|
||||
items: {
|
||||
type: 'json',
|
||||
description:
|
||||
'Vault credentials available to a run [{itemId, connectionId, label, vaultName, domains, fieldMetadata, hasTotp}]',
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
export const TinyFishBlockMeta = {
|
||||
tags: ['web-scraping', 'automation', 'agentic'],
|
||||
url: 'https://www.tinyfish.ai',
|
||||
templates: [
|
||||
{
|
||||
icon: TinyFishIcon,
|
||||
title: 'TinyFish competitor pricing watch',
|
||||
prompt:
|
||||
'Create a scheduled workflow that runs a TinyFish agent weekly against three competitor pricing pages, extracts every plan name and monthly price into a fixed output schema, diffs it against the table from last week, and posts the changes to Slack.',
|
||||
modules: ['scheduled', 'tables', 'agent', 'workflows'],
|
||||
category: 'operations',
|
||||
tags: ['research', 'monitoring'],
|
||||
alsoIntegrations: ['slack'],
|
||||
},
|
||||
{
|
||||
icon: TinyFishIcon,
|
||||
title: 'TinyFish supplier portal collector',
|
||||
prompt:
|
||||
'Build a workflow that uses a TinyFish agent with vault credentials to log into supplier portals that have no API, download the outstanding invoices, and write the invoice metadata to a finance table.',
|
||||
modules: ['scheduled', 'tables', 'files', 'agent', 'workflows'],
|
||||
category: 'operations',
|
||||
tags: ['finance', 'automation'],
|
||||
},
|
||||
{
|
||||
icon: TinyFishIcon,
|
||||
title: 'TinyFish research briefing',
|
||||
prompt:
|
||||
'Create a workflow that uses TinyFish Search to find the top sources on a topic, fetches each one as clean markdown, and has an agent write a cited briefing to a file.',
|
||||
modules: ['files', 'agent', 'workflows'],
|
||||
category: 'marketing',
|
||||
tags: ['research', 'content'],
|
||||
},
|
||||
{
|
||||
icon: TinyFishIcon,
|
||||
title: 'TinyFish lead site enrichment',
|
||||
prompt:
|
||||
'Build a workflow that reads company domains from a table, fetches each homepage and pricing page with TinyFish, extracts positioning and price points into a schema, and writes the enriched rows back.',
|
||||
modules: ['tables', 'agent', 'workflows'],
|
||||
category: 'sales',
|
||||
tags: ['enrichment', 'research'],
|
||||
},
|
||||
{
|
||||
icon: TinyFishIcon,
|
||||
title: 'TinyFish long-running run tracker',
|
||||
prompt:
|
||||
'Create a workflow that queues a TinyFish agent run asynchronously, stores the run ID in a table, and a second scheduled workflow that polls each open run, writes the extracted result back, and cancels runs older than an hour.',
|
||||
modules: ['scheduled', 'tables', 'workflows'],
|
||||
category: 'engineering',
|
||||
tags: ['automation', 'monitoring'],
|
||||
},
|
||||
{
|
||||
icon: TinyFishIcon,
|
||||
title: 'TinyFish signup flow QA',
|
||||
prompt:
|
||||
'Build a workflow that runs a TinyFish agent in strict mode against the staging signup flow every morning, checks that account creation succeeds end to end, and opens a Linear issue with the failing step when it does not.',
|
||||
modules: ['scheduled', 'agent', 'workflows'],
|
||||
category: 'engineering',
|
||||
tags: ['engineering', 'automation'],
|
||||
alsoIntegrations: ['linear'],
|
||||
},
|
||||
{
|
||||
icon: TinyFishIcon,
|
||||
title: 'TinyFish review monitor',
|
||||
prompt:
|
||||
'Create a scheduled workflow that uses TinyFish Search to find new reviews mentioning your product, fetches each review page, classifies sentiment with an agent, and writes notable reviews to a tracking table.',
|
||||
modules: ['scheduled', 'tables', 'agent', 'workflows'],
|
||||
category: 'marketing',
|
||||
tags: ['marketing', 'monitoring'],
|
||||
},
|
||||
{
|
||||
icon: TinyFishIcon,
|
||||
title: 'TinyFish regulatory filing digest',
|
||||
prompt:
|
||||
'Build a workflow that fetches a regulator’s notices page with TinyFish every morning, extracts new filings into a schema, and emails a digest of anything matching your watch list.',
|
||||
modules: ['scheduled', 'tables', 'agent', 'workflows'],
|
||||
category: 'operations',
|
||||
tags: ['research', 'monitoring'],
|
||||
alsoIntegrations: ['gmail'],
|
||||
},
|
||||
],
|
||||
skills: [
|
||||
{
|
||||
name: 'extract-structured-data-from-site',
|
||||
description:
|
||||
'Drive a TinyFish web agent to navigate a site and return data matching a JSON schema. Use to pull records — prices, listings, table rows — from pages that have no API.',
|
||||
content:
|
||||
'# Extract Structured Data From Site\n\nUse the TinyFish Run Agent operation to read a website and return structured data.\n\n## Steps\n1. Set Website URL to the page the agent should start on. Starting closer to the data costs fewer steps.\n2. Write a Goal that names exactly what to collect and where, e.g. "open the pricing page and collect every plan name and monthly price".\n3. Provide an Output Schema (JSON Schema draft-07) describing the fields you want back. TinyFish re-prompts the agent when the result does not match and reports the mismatches in `schemaValidation`.\n4. Raise Max Steps for deeper flows; switch Browser Profile to Stealth when the site blocks automation.\n\n## Output\nReturn the extracted `result` object. Check `schemaValidation.valid` before trusting it, and report any field the agent could not find rather than filling it in.',
|
||||
},
|
||||
{
|
||||
name: 'automate-web-task',
|
||||
description:
|
||||
'Have a TinyFish agent complete a multi-step task on a website — logging in, navigating, filling and submitting forms. Use when a site has no API and a person would normally do the clicks.',
|
||||
content:
|
||||
'# Automate Web Task\n\nUse the TinyFish Run Agent operation to complete a goal-oriented task on the web.\n\n## Steps\n1. Set Website URL to the entry point and write a Goal that states the steps and the success condition, e.g. "log in, open Billing, download the latest invoice".\n2. Turn on Use Vault Credentials when the task needs a login, and scope it with Vault Credential URIs so only the intended credential is available.\n3. Use Agent Mode "strict" when the run is a test that should fail fast instead of improvising.\n4. Enable Use Proxy and pick a Proxy Country when the site is geo-restricted.\n\n## Output\nReport whether the run completed, what the agent extracted, and the step count. On failure, quote the `error` message and category — AGENT_FAILURE means the goal needs rewording, SYSTEM_FAILURE is worth retrying.',
|
||||
},
|
||||
{
|
||||
name: 'search-and-read-the-web',
|
||||
description:
|
||||
'Search the web with TinyFish and fetch the winning pages as clean markdown. Use to gather current sources before writing or answering.',
|
||||
content:
|
||||
'# Search And Read The Web\n\nPair the TinyFish Search and Fetch URLs operations to gather sources.\n\n## Steps\n1. Run Search with a specific Query. Set Location and Language when the answer is regional.\n2. Pick the result URLs worth reading — position and snippet tell you which.\n3. Run Fetch URLs with up to 10 of those URLs and Format "markdown". Turn on Extract Links when you need to follow deeper pages.\n4. Per-URL failures land in `errors` and never fail the whole call, so check it before assuming a page was read.\n\n## Output\nSummarize from the fetched text and cite each claim with the source URL. Say which URLs failed instead of quietly dropping them.',
|
||||
},
|
||||
{
|
||||
name: 'scope-a-run-to-one-vault-credential',
|
||||
description:
|
||||
'Find the TinyFish vault credential URI for a site and scope a single agent run to it. Use when a run needs one specific login and must not see the rest of the vault.',
|
||||
content:
|
||||
'# Scope A Run To One Vault Credential\n\nUse List Vault Items to discover credential URIs, then pass one to a run.\n\n## Steps\n1. Run the List Vault Items operation. It returns display-safe metadata only — `itemId`, `label`, `vaultName`, `domains`, and whether the credential carries a TOTP. No secret values ever leave TinyFish.\n2. Pick the item whose `domains` match the site you are automating and copy its `itemId` (it looks like `cred:conn-abc:Work:item-123`).\n3. On the Run Agent operation, turn on Use Vault Credentials and paste that `itemId` into Vault Credential URIs. Leaving the field empty exposes every enabled item to the run instead.\n4. Write the goal to reference the login by intent — "sign in and open Billing" — not by pasting any credential.\n\n## Output\nReport which credential the run used by label, and whether the login succeeded. Never echo credential values; they are not returned and must not be reconstructed. If `hasTotp` is false and the site demands a second factor, say so rather than retrying.',
|
||||
},
|
||||
{
|
||||
name: 'diagnose-a-failed-run',
|
||||
description:
|
||||
'Read a failed TinyFish run and decide whether to retry, reword the goal, or escalate. Use when an automation returns FAILED or a workflow keeps burning steps without a result.',
|
||||
content:
|
||||
'# Diagnose A Failed Run\n\nA failed automation comes back as a normal 200 response with the failure inside the run, so read `status` before trusting `result`.\n\n## Steps\n1. Read `error.category`. `AGENT_FAILURE` means the goal or the page is the problem — reword the goal or start the run closer to the target. `SYSTEM_FAILURE` is TinyFish-side; wait `error.retryAfter` seconds and retry the same input. `BILLING_FAILURE` means the TinyFish wallet is empty and no retry will help. `UNKNOWN` should be treated as retryable once.\n2. Compare `numOfSteps` against the Max Steps you set. Hitting the cap means the agent was still working, so raise the cap or narrow the goal.\n3. If an Output Schema was set, read `schemaValidation.errors` — a run can reach the right page and still fail on one mistyped field, and `rePromptAttempts` shows how hard TinyFish already tried to repair it.\n4. For an async run, call Get Run and read the `steps` list to find the last action before the failure. `videoUrl` gives a recording, but the link expires 15 minutes after it is issued.\n\n## Output\nState the category, the concrete cause, and the single next action. Quote `error.message` rather than paraphrasing it, and do not retry a `BILLING_FAILURE` or an `AGENT_FAILURE` without changing the input first.',
|
||||
},
|
||||
{
|
||||
name: 'queue-and-track-long-runs',
|
||||
description:
|
||||
'Queue a TinyFish agent run without waiting, then poll or cancel it by ID. Use for automations too long to hold a workflow step open.',
|
||||
content:
|
||||
'# Queue And Track Long Runs\n\nUse Start Agent Run, then Get Run, Cancel Run, and List Runs to manage work asynchronously.\n\n## Steps\n1. Start Agent Run with the URL and Goal. It returns a `runId` immediately. Set a Webhook URL if something should be notified on completion.\n2. Store the `runId`, then call Get Run to read `status`, `result`, and the step history. `numOfSteps` stays null while the run is in progress.\n3. Call Cancel Run to stop a queued or running automation. It is idempotent — a run that already finished comes back with its terminal status and a message saying so.\n4. Use List Runs with a status or goal filter to find runs you did not record the ID for.\n\n## Output\nReport the run status and, once COMPLETED, the extracted `result`. Note that these operations always need your own TinyFish API key — they are not covered by Sim’s hosted key.',
|
||||
},
|
||||
],
|
||||
} as const satisfies BlockMeta
|
||||
@@ -328,6 +328,7 @@ import { ThinkingBlock } from '@/blocks/blocks/thinking'
|
||||
import { ThriveBlock, ThriveBlockMeta } from '@/blocks/blocks/thrive'
|
||||
import { TikTokBlock, TikTokBlockMeta } from '@/blocks/blocks/tiktok'
|
||||
import { TinybirdBlock, TinybirdBlockMeta } from '@/blocks/blocks/tinybird'
|
||||
import { TinyFishBlock, TinyFishBlockMeta } from '@/blocks/blocks/tinyfish'
|
||||
import { TranslateBlock } from '@/blocks/blocks/translate'
|
||||
import { TrelloBlock, TrelloBlockMeta } from '@/blocks/blocks/trello'
|
||||
import { TriggerDevBlock, TriggerDevBlockMeta } from '@/blocks/blocks/trigger_dev'
|
||||
@@ -672,6 +673,7 @@ export const BLOCK_REGISTRY: Record<string, BlockConfig> = {
|
||||
thrive: ThriveBlock,
|
||||
tiktok: TikTokBlock,
|
||||
tinybird: TinybirdBlock,
|
||||
tinyfish: TinyFishBlock,
|
||||
translate: TranslateBlock,
|
||||
trello: TrelloBlock,
|
||||
trigger_dev: TriggerDevBlock,
|
||||
@@ -971,6 +973,7 @@ export const BLOCK_META_REGISTRY: Record<string, BlockMeta> = {
|
||||
thrive: ThriveBlockMeta,
|
||||
tiktok: TikTokBlockMeta,
|
||||
tinybird: TinybirdBlockMeta,
|
||||
tinyfish: TinyFishBlockMeta,
|
||||
trello: TrelloBlockMeta,
|
||||
trigger_dev: TriggerDevBlockMeta,
|
||||
twilio_sms: TwilioSMSBlockMeta,
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -18,6 +18,7 @@ export const byokProviderIdSchema = z.enum([
|
||||
'firecrawl',
|
||||
'exa',
|
||||
'context_dev',
|
||||
'tinyfish',
|
||||
'serper',
|
||||
'linkup',
|
||||
'perplexity',
|
||||
|
||||
@@ -314,6 +314,7 @@ export const DOCS_MANIFEST: readonly string[] = [
|
||||
'integrations/thrive.mdx',
|
||||
'integrations/tiktok.mdx',
|
||||
'integrations/tinybird.mdx',
|
||||
'integrations/tinyfish.mdx',
|
||||
'integrations/trello-service-account.mdx',
|
||||
'integrations/trello.mdx',
|
||||
'integrations/trigger_dev.mdx',
|
||||
|
||||
@@ -242,6 +242,7 @@ import {
|
||||
ThriveIcon,
|
||||
TikTokIcon,
|
||||
TinybirdIcon,
|
||||
TinyFishIcon,
|
||||
TrelloIcon,
|
||||
TriggerDevIcon,
|
||||
TwilioIcon,
|
||||
@@ -529,6 +530,7 @@ export const blockTypeToIconMap: Record<string, IconComponent> = {
|
||||
thrive: ThriveIcon,
|
||||
tiktok: TikTokIcon,
|
||||
tinybird: TinybirdIcon,
|
||||
tinyfish: TinyFishIcon,
|
||||
trello: TrelloIcon,
|
||||
trigger_dev: TriggerDevIcon,
|
||||
twilio: TwilioIcon,
|
||||
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -5131,6 +5131,16 @@ import {
|
||||
tinybirdQueryTool,
|
||||
tinybirdTruncateDatasourceTool,
|
||||
} from '@/tools/tinybird'
|
||||
import {
|
||||
tinyfishCancelRunTool,
|
||||
tinyfishFetchTool,
|
||||
tinyfishGetRunTool,
|
||||
tinyfishListRunsTool,
|
||||
tinyfishListVaultItemsTool,
|
||||
tinyfishRunAsyncTool,
|
||||
tinyfishRunTool,
|
||||
tinyfishSearchTool,
|
||||
} from '@/tools/tinyfish'
|
||||
import {
|
||||
trelloAddChecklistItemTool,
|
||||
trelloAddChecklistTool,
|
||||
@@ -9090,6 +9100,14 @@ export const tools: Record<string, ToolConfig> = {
|
||||
tinybird_truncate_datasource: tinybirdTruncateDatasourceTool,
|
||||
tinybird_delete_datasource_rows: tinybirdDeleteDatasourceRowsTool,
|
||||
tinybird_get_job: tinybirdGetJobTool,
|
||||
tinyfish_run: tinyfishRunTool,
|
||||
tinyfish_run_async: tinyfishRunAsyncTool,
|
||||
tinyfish_get_run: tinyfishGetRunTool,
|
||||
tinyfish_cancel_run: tinyfishCancelRunTool,
|
||||
tinyfish_list_runs: tinyfishListRunsTool,
|
||||
tinyfish_search: tinyfishSearchTool,
|
||||
tinyfish_fetch: tinyfishFetchTool,
|
||||
tinyfish_list_vault_items: tinyfishListVaultItemsTool,
|
||||
stagehand_extract: stagehandExtractTool,
|
||||
stagehand_agent: stagehandAgentTool,
|
||||
mem0_add_memories: mem0AddMemoriesTool,
|
||||
|
||||
@@ -0,0 +1,83 @@
|
||||
import type {
|
||||
TinyFishCancelRunParams,
|
||||
TinyFishCancelRunResponse,
|
||||
TinyFishRawCancel,
|
||||
} from '@/tools/tinyfish/types'
|
||||
import {
|
||||
TINYFISH_AGENT_API_BASE,
|
||||
tinyfishErrorMessage,
|
||||
tinyfishHeaders,
|
||||
} from '@/tools/tinyfish/utils'
|
||||
import type { ToolConfig } from '@/tools/types'
|
||||
|
||||
/**
|
||||
* Cancels a queued or running automation.
|
||||
*
|
||||
* TinyFish can only cancel runs started through the async or SSE endpoints; a
|
||||
* run started synchronously has no cancellable handle.
|
||||
*/
|
||||
export const cancelRunTool: ToolConfig<TinyFishCancelRunParams, TinyFishCancelRunResponse> = {
|
||||
id: 'tinyfish_cancel_run',
|
||||
name: 'TinyFish Cancel Run',
|
||||
description: 'Cancel a queued or in-progress TinyFish automation run by its ID',
|
||||
version: '1.0.0',
|
||||
|
||||
params: {
|
||||
runId: {
|
||||
type: 'string',
|
||||
required: true,
|
||||
visibility: 'user-or-llm',
|
||||
description: 'Identifier of the run to cancel',
|
||||
},
|
||||
apiKey: {
|
||||
type: 'string',
|
||||
required: true,
|
||||
visibility: 'user-only',
|
||||
description: 'TinyFish API key',
|
||||
},
|
||||
},
|
||||
|
||||
request: {
|
||||
url: (params) =>
|
||||
`${TINYFISH_AGENT_API_BASE}/v1/runs/${encodeURIComponent(params.runId.trim())}/cancel`,
|
||||
method: 'POST',
|
||||
headers: (params) => tinyfishHeaders(params.apiKey),
|
||||
},
|
||||
|
||||
transformResponse: async (response: Response) => {
|
||||
if (!response.ok) {
|
||||
throw new Error(await tinyfishErrorMessage(response))
|
||||
}
|
||||
|
||||
const data = (await response.json()) as TinyFishRawCancel
|
||||
|
||||
return {
|
||||
success: true,
|
||||
output: {
|
||||
runId: data.run_id ?? '',
|
||||
status: data.status ?? 'CANCELLED',
|
||||
cancelledAt: data.cancelled_at ?? null,
|
||||
message: data.message ?? null,
|
||||
},
|
||||
}
|
||||
},
|
||||
|
||||
outputs: {
|
||||
runId: { type: 'string', description: 'Run identifier' },
|
||||
status: {
|
||||
type: 'string',
|
||||
description:
|
||||
'Status after the call: CANCELLED, or the terminal status the run already reached',
|
||||
},
|
||||
cancelledAt: {
|
||||
type: 'string',
|
||||
description: 'ISO 8601 timestamp of the cancellation, null when nothing was cancelled',
|
||||
optional: true,
|
||||
},
|
||||
message: {
|
||||
type: 'string',
|
||||
description: 'Context such as "Run already cancelled" or "Run already finished"',
|
||||
optional: true,
|
||||
},
|
||||
},
|
||||
}
|
||||
@@ -0,0 +1,162 @@
|
||||
import { tinyfishFetchHosting } from '@/tools/tinyfish/hosting'
|
||||
import type {
|
||||
TinyFishFetchParams,
|
||||
TinyFishFetchResponse,
|
||||
TinyFishRawFetch,
|
||||
} from '@/tools/tinyfish/types'
|
||||
import {
|
||||
MAX_FETCH_URLS,
|
||||
parseList,
|
||||
TINYFISH_FETCH_API_BASE,
|
||||
tinyfishErrorMessage,
|
||||
tinyfishHeaders,
|
||||
} from '@/tools/tinyfish/utils'
|
||||
import type { ToolConfig } from '@/tools/types'
|
||||
|
||||
export const fetchUrlsTool: ToolConfig<TinyFishFetchParams, TinyFishFetchResponse> = {
|
||||
id: 'tinyfish_fetch',
|
||||
name: 'TinyFish Fetch',
|
||||
description:
|
||||
'Fetch up to 10 URLs with TinyFish, rendering JavaScript when needed, and return clean extracted content',
|
||||
version: '1.0.0',
|
||||
|
||||
params: {
|
||||
urls: {
|
||||
type: 'string',
|
||||
required: true,
|
||||
visibility: 'user-or-llm',
|
||||
description: 'Comma-separated list of 1-10 URLs to fetch',
|
||||
},
|
||||
format: {
|
||||
type: 'string',
|
||||
required: false,
|
||||
visibility: 'user-or-llm',
|
||||
description: 'Extraction format: "markdown" (default), "html", or "json"',
|
||||
},
|
||||
links: {
|
||||
type: 'boolean',
|
||||
required: false,
|
||||
visibility: 'user-or-llm',
|
||||
description: 'Also return every outbound link found on each page',
|
||||
},
|
||||
imageLinks: {
|
||||
type: 'boolean',
|
||||
required: false,
|
||||
visibility: 'user-or-llm',
|
||||
description: 'Also return every image URL found on each page',
|
||||
},
|
||||
apiKey: {
|
||||
type: 'string',
|
||||
required: true,
|
||||
visibility: 'user-only',
|
||||
description: 'TinyFish API key',
|
||||
},
|
||||
},
|
||||
|
||||
hosting: tinyfishFetchHosting(),
|
||||
|
||||
request: {
|
||||
url: `${TINYFISH_FETCH_API_BASE}/`,
|
||||
method: 'POST',
|
||||
headers: (params) => tinyfishHeaders(params.apiKey),
|
||||
body: (params) => {
|
||||
const urls = parseList(params.urls)
|
||||
/**
|
||||
* TinyFish rejects an empty or over-long list with a generic 400, so the
|
||||
* documented 1-10 bound is checked here to name the actual problem.
|
||||
*/
|
||||
if (urls.length === 0) {
|
||||
throw new Error('At least one URL is required')
|
||||
}
|
||||
if (urls.length > MAX_FETCH_URLS) {
|
||||
throw new Error(`TinyFish Fetch accepts at most ${MAX_FETCH_URLS} URLs per request`)
|
||||
}
|
||||
|
||||
const body: Record<string, unknown> = { urls }
|
||||
if (params.format) body.format = params.format
|
||||
if (params.links) body.links = true
|
||||
if (params.imageLinks) body.image_links = true
|
||||
return body
|
||||
},
|
||||
},
|
||||
|
||||
transformResponse: async (response: Response) => {
|
||||
if (!response.ok) {
|
||||
throw new Error(await tinyfishErrorMessage(response))
|
||||
}
|
||||
|
||||
const data = (await response.json()) as TinyFishRawFetch
|
||||
|
||||
return {
|
||||
success: true,
|
||||
output: {
|
||||
results: (data.results ?? []).map((result) => ({
|
||||
url: result?.url ?? '',
|
||||
finalUrl: result?.final_url ?? null,
|
||||
title: result?.title ?? null,
|
||||
description: result?.description ?? null,
|
||||
language: result?.language ?? null,
|
||||
format: result?.format ?? 'markdown',
|
||||
text: result?.text ?? null,
|
||||
author: result?.author ?? null,
|
||||
publishedDate: result?.published_date ?? null,
|
||||
links: result?.links ?? [],
|
||||
imageLinks: result?.image_links ?? [],
|
||||
latencyMs: result?.latency_ms ?? null,
|
||||
})),
|
||||
errors: (data.errors ?? []).map((issue) => ({
|
||||
url: issue?.url ?? '',
|
||||
error: issue?.error ?? '',
|
||||
})),
|
||||
},
|
||||
}
|
||||
},
|
||||
|
||||
outputs: {
|
||||
results: {
|
||||
type: 'array',
|
||||
description: 'Successfully fetched pages',
|
||||
items: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
url: { type: 'string', description: 'URL that was requested' },
|
||||
finalUrl: { type: 'string', description: 'URL after redirects', optional: true },
|
||||
title: { type: 'string', description: 'Page title', optional: true },
|
||||
description: { type: 'string', description: 'Meta description', optional: true },
|
||||
language: { type: 'string', description: 'Detected language code', optional: true },
|
||||
format: { type: 'string', description: 'Format of the extracted content' },
|
||||
text: {
|
||||
type: 'json',
|
||||
description:
|
||||
'Extracted content — a string for markdown and html, a document tree for json',
|
||||
optional: true,
|
||||
},
|
||||
author: { type: 'string', description: 'Page author', optional: true },
|
||||
publishedDate: { type: 'string', description: 'Published date', optional: true },
|
||||
links: {
|
||||
type: 'array',
|
||||
description: 'Outbound links, only when links was requested',
|
||||
items: { type: 'string', description: 'Absolute link URL' },
|
||||
},
|
||||
imageLinks: {
|
||||
type: 'array',
|
||||
description: 'Image URLs, only when image links were requested',
|
||||
items: { type: 'string', description: 'Absolute image URL' },
|
||||
},
|
||||
latencyMs: { type: 'number', description: 'Fetch latency in ms', optional: true },
|
||||
},
|
||||
},
|
||||
},
|
||||
errors: {
|
||||
type: 'array',
|
||||
description: 'URLs that failed. A per-URL failure never fails the whole request',
|
||||
items: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
url: { type: 'string', description: 'URL that failed' },
|
||||
error: { type: 'string', description: 'Why the fetch failed' },
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
@@ -0,0 +1,89 @@
|
||||
import {
|
||||
RUN_SUMMARY_OUTPUT_PROPERTIES,
|
||||
type TinyFishGetRunParams,
|
||||
type TinyFishGetRunResponse,
|
||||
type TinyFishRawRunDetail,
|
||||
} from '@/tools/tinyfish/types'
|
||||
import {
|
||||
mapRunSummary,
|
||||
TINYFISH_AGENT_API_BASE,
|
||||
tinyfishErrorMessage,
|
||||
tinyfishHeaders,
|
||||
} from '@/tools/tinyfish/utils'
|
||||
import type { ToolConfig } from '@/tools/types'
|
||||
|
||||
export const getRunTool: ToolConfig<TinyFishGetRunParams, TinyFishGetRunResponse> = {
|
||||
id: 'tinyfish_get_run',
|
||||
name: 'TinyFish Get Run',
|
||||
description:
|
||||
'Get the status, extracted result, and step history of a TinyFish automation run by its ID',
|
||||
version: '1.0.0',
|
||||
|
||||
params: {
|
||||
runId: {
|
||||
type: 'string',
|
||||
required: true,
|
||||
visibility: 'user-or-llm',
|
||||
description: 'Identifier of the run to look up',
|
||||
},
|
||||
apiKey: {
|
||||
type: 'string',
|
||||
required: true,
|
||||
visibility: 'user-only',
|
||||
description: 'TinyFish API key',
|
||||
},
|
||||
},
|
||||
|
||||
request: {
|
||||
url: (params) =>
|
||||
`${TINYFISH_AGENT_API_BASE}/v1/runs/${encodeURIComponent(params.runId.trim())}`,
|
||||
method: 'GET',
|
||||
headers: (params) => tinyfishHeaders(params.apiKey),
|
||||
},
|
||||
|
||||
transformResponse: async (response: Response) => {
|
||||
if (!response.ok) {
|
||||
throw new Error(await tinyfishErrorMessage(response))
|
||||
}
|
||||
|
||||
const data = (await response.json()) as TinyFishRawRunDetail
|
||||
|
||||
return {
|
||||
success: true,
|
||||
output: {
|
||||
...mapRunSummary(data),
|
||||
videoUrl: data.video_url ?? null,
|
||||
steps: (data.steps ?? []).map((step) => ({
|
||||
id: step?.id ?? '',
|
||||
timestamp: step?.timestamp ?? '',
|
||||
status: step?.status ?? 'PENDING',
|
||||
action: step?.action ?? null,
|
||||
duration: step?.duration ?? null,
|
||||
})),
|
||||
},
|
||||
}
|
||||
},
|
||||
|
||||
outputs: {
|
||||
...RUN_SUMMARY_OUTPUT_PROPERTIES,
|
||||
videoUrl: {
|
||||
type: 'string',
|
||||
description: 'Presigned recording URL, expires 15 minutes after it is issued',
|
||||
optional: true,
|
||||
},
|
||||
steps: {
|
||||
type: 'array',
|
||||
description: 'Steps the agent took during the run',
|
||||
items: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
id: { type: 'string', description: 'Step identifier' },
|
||||
timestamp: { type: 'string', description: 'ISO 8601 timestamp of the step' },
|
||||
status: { type: 'string', description: 'Status of the run at this step' },
|
||||
action: { type: 'string', description: 'Action the agent took', optional: true },
|
||||
duration: { type: 'string', description: 'Time the step took', optional: true },
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
@@ -0,0 +1,139 @@
|
||||
import { parseList } from '@/tools/tinyfish/utils'
|
||||
import type { ToolHostingConfig } from '@/tools/types'
|
||||
|
||||
/** Env var prefix for TinyFish hosted keys. One key serves every TinyFish surface. */
|
||||
export const TINYFISH_API_KEY_PREFIX = 'TINYFISH_API_KEY'
|
||||
|
||||
/**
|
||||
* Dollar cost of a single TinyFish Agent step.
|
||||
*
|
||||
* TinyFish bills the Agent product per step from a prepaid wallet — there are
|
||||
* no plan tiers, so this rate is the same for every hosted key.
|
||||
*
|
||||
* Source: https://www.tinyfish.ai/pricing
|
||||
*/
|
||||
export const TINYFISH_AGENT_STEP_USD = 0.016
|
||||
|
||||
/**
|
||||
* Hosting config for the synchronous Agent run.
|
||||
*
|
||||
* The run response reports `num_of_steps`, which is the exact unit TinyFish
|
||||
* bills on, so the charge always comes from what the API reported rather than
|
||||
* an estimate.
|
||||
*
|
||||
* Two limits of this config are deliberate and worth knowing before raising it:
|
||||
*
|
||||
* - TinyFish's real Agent ceiling is 2 concurrent runs per account, and the
|
||||
* token bucket has no concurrency dimension. `requestsPerMinute` is a
|
||||
* proxy, not an equivalent: a synchronous run lasts minutes, so five per
|
||||
* minute already permits more in-flight runs than the account allows. The
|
||||
* value is kept low for that reason, but hitting the account ceiling
|
||||
* surfaces as a TinyFish 429 rather than a Sim-side wait.
|
||||
* - Sim meters only successful executions, so a run that ends `FAILED` is
|
||||
* unbilled even though TinyFish charged the hosted wallet for every step it
|
||||
* took. Closing that needs a change to the executor's success gate, not to
|
||||
* this pricing function.
|
||||
*/
|
||||
export function tinyfishAgentHosting<P>(): ToolHostingConfig<P> {
|
||||
return {
|
||||
envKeyPrefix: TINYFISH_API_KEY_PREFIX,
|
||||
apiKeyParam: 'apiKey',
|
||||
byokProviderId: 'tinyfish',
|
||||
pricing: {
|
||||
type: 'custom',
|
||||
getCost: (_params, output) => {
|
||||
const reported = output.numOfSteps
|
||||
if (reported == null) {
|
||||
throw new Error('TinyFish run response missing num_of_steps')
|
||||
}
|
||||
|
||||
const numOfSteps = Number(reported)
|
||||
if (!Number.isFinite(numOfSteps)) {
|
||||
throw new Error('TinyFish run response returned a non-numeric num_of_steps')
|
||||
}
|
||||
|
||||
return {
|
||||
cost: numOfSteps * TINYFISH_AGENT_STEP_USD,
|
||||
metadata: { steps: numOfSteps },
|
||||
}
|
||||
},
|
||||
},
|
||||
rateLimit: {
|
||||
mode: 'per_request',
|
||||
requestsPerMinute: 5,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Hosting config for the Search API.
|
||||
*
|
||||
* Search is free at any wallet balance, so the hosted key costs nothing to run.
|
||||
* It is still worth hosting: free is not unauthenticated — the endpoint requires
|
||||
* an `X-API-Key`, so without a hosted key a user would have to create a TinyFish
|
||||
* account before they could search at all.
|
||||
*
|
||||
* Because nothing is billed, the rate limit is the only backpressure. TinyFish
|
||||
* enforces 30 requests/minute **per API key**, so the numbered key pool raises
|
||||
* the ceiling proportionally, and this per-workspace share lets roughly three
|
||||
* workspaces run flat out against a single key.
|
||||
*
|
||||
* Source: https://docs.tinyfish.ai/search-api/reference
|
||||
*/
|
||||
export function tinyfishSearchHosting<P>(): ToolHostingConfig<P> {
|
||||
return {
|
||||
envKeyPrefix: TINYFISH_API_KEY_PREFIX,
|
||||
apiKeyParam: 'apiKey',
|
||||
byokProviderId: 'tinyfish',
|
||||
pricing: {
|
||||
type: 'per_request',
|
||||
cost: 0,
|
||||
},
|
||||
rateLimit: {
|
||||
mode: 'per_request',
|
||||
requestsPerMinute: 10,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Hosting config for the Fetch API.
|
||||
*
|
||||
* Fetch is free at any wallet balance and is hosted for the same reason as Search:
|
||||
* the endpoint still requires an API key, so hosting is what makes it work without
|
||||
* the user holding a TinyFish account.
|
||||
*
|
||||
* Its documented ceiling is measured in URLs (150/minute **per API key**), not in
|
||||
* requests, and one request carries up to 10 URLs. The URL count is therefore
|
||||
* tracked as its own dimension so a workspace batching 10 URLs per call is
|
||||
* throttled on the same axis TinyFish enforces. Usage is read from the submitted
|
||||
* list rather than the returned arrays, because TinyFish counts a URL it could not
|
||||
* fetch and the response would undercount one that landed in neither array.
|
||||
*
|
||||
* The 40/minute per-workspace share leaves room for roughly three workspaces to
|
||||
* run flat out against a single key.
|
||||
*
|
||||
* Source: https://docs.tinyfish.ai/fetch-api/reference
|
||||
*/
|
||||
export function tinyfishFetchHosting<P>(): ToolHostingConfig<P> {
|
||||
return {
|
||||
envKeyPrefix: TINYFISH_API_KEY_PREFIX,
|
||||
apiKeyParam: 'apiKey',
|
||||
byokProviderId: 'tinyfish',
|
||||
pricing: {
|
||||
type: 'per_request',
|
||||
cost: 0,
|
||||
},
|
||||
rateLimit: {
|
||||
mode: 'custom',
|
||||
requestsPerMinute: 10,
|
||||
dimensions: [
|
||||
{
|
||||
name: 'urls',
|
||||
limitPerMinute: 40,
|
||||
extractUsage: (params) => parseList(params.urls as string | string[] | undefined).length,
|
||||
},
|
||||
],
|
||||
},
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
import { cancelRunTool } from '@/tools/tinyfish/cancel_run'
|
||||
import { fetchUrlsTool } from '@/tools/tinyfish/fetch_urls'
|
||||
import { getRunTool } from '@/tools/tinyfish/get_run'
|
||||
import { listRunsTool } from '@/tools/tinyfish/list_runs'
|
||||
import { listVaultItemsTool } from '@/tools/tinyfish/list_vault_items'
|
||||
import { runTool } from '@/tools/tinyfish/run'
|
||||
import { runAsyncTool } from '@/tools/tinyfish/run_async'
|
||||
import { searchTool } from '@/tools/tinyfish/search'
|
||||
|
||||
export const tinyfishCancelRunTool = cancelRunTool
|
||||
export const tinyfishFetchTool = fetchUrlsTool
|
||||
export const tinyfishGetRunTool = getRunTool
|
||||
export const tinyfishListRunsTool = listRunsTool
|
||||
export const tinyfishListVaultItemsTool = listVaultItemsTool
|
||||
export const tinyfishRunTool = runTool
|
||||
export const tinyfishRunAsyncTool = runAsyncTool
|
||||
export const tinyfishSearchTool = searchTool
|
||||
@@ -0,0 +1,128 @@
|
||||
import {
|
||||
RUN_SUMMARY_OUTPUT_PROPERTIES,
|
||||
type TinyFishListRunsParams,
|
||||
type TinyFishListRunsResponse,
|
||||
type TinyFishRawRunList,
|
||||
} from '@/tools/tinyfish/types'
|
||||
import {
|
||||
mapRunSummary,
|
||||
TINYFISH_AGENT_API_BASE,
|
||||
tinyfishErrorMessage,
|
||||
tinyfishHeaders,
|
||||
} from '@/tools/tinyfish/utils'
|
||||
import type { ToolConfig } from '@/tools/types'
|
||||
|
||||
export const listRunsTool: ToolConfig<TinyFishListRunsParams, TinyFishListRunsResponse> = {
|
||||
id: 'tinyfish_list_runs',
|
||||
name: 'TinyFish List Runs',
|
||||
description:
|
||||
'List TinyFish automation runs, optionally filtered by status, goal text, or creation date',
|
||||
version: '1.0.0',
|
||||
|
||||
params: {
|
||||
status: {
|
||||
type: 'string',
|
||||
required: false,
|
||||
visibility: 'user-or-llm',
|
||||
description: 'Filter by run status: PENDING, RUNNING, COMPLETED, FAILED, or CANCELLED',
|
||||
},
|
||||
goal: {
|
||||
type: 'string',
|
||||
required: false,
|
||||
visibility: 'user-or-llm',
|
||||
description: 'Filter by goal text (case-insensitive partial match, max 500 characters)',
|
||||
},
|
||||
createdAfter: {
|
||||
type: 'string',
|
||||
required: false,
|
||||
visibility: 'user-or-llm',
|
||||
description: 'Only return runs created after this ISO 8601 timestamp',
|
||||
},
|
||||
createdBefore: {
|
||||
type: 'string',
|
||||
required: false,
|
||||
visibility: 'user-or-llm',
|
||||
description: 'Only return runs created before this ISO 8601 timestamp',
|
||||
},
|
||||
sortDirection: {
|
||||
type: 'string',
|
||||
required: false,
|
||||
visibility: 'user-or-llm',
|
||||
description: 'Sort by creation time: "desc" (newest first, default) or "asc"',
|
||||
},
|
||||
limit: {
|
||||
type: 'number',
|
||||
required: false,
|
||||
visibility: 'user-or-llm',
|
||||
description: 'Maximum runs to return (1-100, default 20)',
|
||||
},
|
||||
cursor: {
|
||||
type: 'string',
|
||||
required: false,
|
||||
visibility: 'user-or-llm',
|
||||
description: 'Pagination cursor returned by a previous call',
|
||||
},
|
||||
apiKey: {
|
||||
type: 'string',
|
||||
required: true,
|
||||
visibility: 'user-only',
|
||||
description: 'TinyFish API key',
|
||||
},
|
||||
},
|
||||
|
||||
request: {
|
||||
url: (params) => {
|
||||
const query = new URLSearchParams()
|
||||
if (params.status) query.set('status', params.status)
|
||||
if (params.goal) query.set('goal', params.goal)
|
||||
if (params.createdAfter) query.set('created_after', params.createdAfter)
|
||||
if (params.createdBefore) query.set('created_before', params.createdBefore)
|
||||
if (params.sortDirection) query.set('sort_direction', params.sortDirection)
|
||||
if (params.cursor) query.set('cursor', params.cursor)
|
||||
if (typeof params.limit === 'number' && Number.isFinite(params.limit)) {
|
||||
query.set('limit', String(params.limit))
|
||||
}
|
||||
|
||||
const search = query.toString()
|
||||
return `${TINYFISH_AGENT_API_BASE}/v1/runs${search ? `?${search}` : ''}`
|
||||
},
|
||||
method: 'GET',
|
||||
headers: (params) => tinyfishHeaders(params.apiKey),
|
||||
},
|
||||
|
||||
transformResponse: async (response: Response) => {
|
||||
if (!response.ok) {
|
||||
throw new Error(await tinyfishErrorMessage(response))
|
||||
}
|
||||
|
||||
const data = (await response.json()) as TinyFishRawRunList
|
||||
|
||||
return {
|
||||
success: true,
|
||||
output: {
|
||||
runs: (data.data ?? []).map(mapRunSummary),
|
||||
total: data.pagination?.total ?? 0,
|
||||
nextCursor: data.pagination?.next_cursor ?? null,
|
||||
hasMore: data.pagination?.has_more ?? false,
|
||||
},
|
||||
}
|
||||
},
|
||||
|
||||
outputs: {
|
||||
runs: {
|
||||
type: 'array',
|
||||
description: 'Runs matching the filters, newest first by default',
|
||||
items: {
|
||||
type: 'object',
|
||||
properties: RUN_SUMMARY_OUTPUT_PROPERTIES,
|
||||
},
|
||||
},
|
||||
total: { type: 'number', description: 'Total runs matching the filters' },
|
||||
nextCursor: {
|
||||
type: 'string',
|
||||
description: 'Cursor for the next page, null when there are no more results',
|
||||
optional: true,
|
||||
},
|
||||
hasMore: { type: 'boolean', description: 'Whether more results follow this page' },
|
||||
},
|
||||
}
|
||||
@@ -0,0 +1,111 @@
|
||||
import type {
|
||||
TinyFishListVaultItemsParams,
|
||||
TinyFishListVaultItemsResponse,
|
||||
TinyFishRawVaultItems,
|
||||
} from '@/tools/tinyfish/types'
|
||||
import {
|
||||
TINYFISH_AGENT_API_BASE,
|
||||
tinyfishErrorMessage,
|
||||
tinyfishHeaders,
|
||||
} from '@/tools/tinyfish/utils'
|
||||
import type { ToolConfig } from '@/tools/types'
|
||||
|
||||
/**
|
||||
* Lists the credentials a connected password manager exposes to TinyFish.
|
||||
*
|
||||
* The response is display-safe metadata only — it carries the credential URIs an
|
||||
* automation run scopes itself to, never the secret values behind them.
|
||||
*/
|
||||
export const listVaultItemsTool: ToolConfig<
|
||||
TinyFishListVaultItemsParams,
|
||||
TinyFishListVaultItemsResponse
|
||||
> = {
|
||||
id: 'tinyfish_list_vault_items',
|
||||
name: 'TinyFish List Vault Items',
|
||||
description:
|
||||
'List the credentials available from password managers connected to TinyFish, with the URIs an agent run can be scoped to',
|
||||
version: '1.0.0',
|
||||
|
||||
params: {
|
||||
apiKey: {
|
||||
type: 'string',
|
||||
required: true,
|
||||
visibility: 'user-only',
|
||||
description: 'TinyFish API key',
|
||||
},
|
||||
},
|
||||
|
||||
request: {
|
||||
url: `${TINYFISH_AGENT_API_BASE}/v1/vault/items`,
|
||||
method: 'GET',
|
||||
headers: (params) => tinyfishHeaders(params.apiKey),
|
||||
},
|
||||
|
||||
transformResponse: async (response: Response) => {
|
||||
if (!response.ok) {
|
||||
throw new Error(await tinyfishErrorMessage(response))
|
||||
}
|
||||
|
||||
const data = (await response.json()) as TinyFishRawVaultItems
|
||||
|
||||
return {
|
||||
success: true,
|
||||
output: {
|
||||
items: (data.items ?? []).map((item) => ({
|
||||
itemId: item?.itemId ?? '',
|
||||
connectionId: item?.connectionId ?? null,
|
||||
label: item?.label ?? '',
|
||||
vaultName: item?.vaultName ?? '',
|
||||
domains: item?.domains ?? [],
|
||||
fieldMetadata: (item?.fieldMetadata ?? []).map((field) => ({
|
||||
fieldId: field?.fieldId ?? '',
|
||||
label: field?.label ?? '',
|
||||
type: field?.type ?? 'STRING',
|
||||
})),
|
||||
hasTotp: item?.hasTotp ?? false,
|
||||
})),
|
||||
},
|
||||
}
|
||||
},
|
||||
|
||||
outputs: {
|
||||
items: {
|
||||
type: 'array',
|
||||
description: 'Credentials available to automation runs',
|
||||
items: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
itemId: {
|
||||
type: 'string',
|
||||
description: 'Credential URI, used as a Vault Credential URI on a run',
|
||||
},
|
||||
connectionId: {
|
||||
type: 'string',
|
||||
description: 'Identifier of the vault connection it came from',
|
||||
optional: true,
|
||||
},
|
||||
label: { type: 'string', description: 'Credential name, such as "Amazon Login"' },
|
||||
vaultName: { type: 'string', description: 'Vault the credential lives in' },
|
||||
domains: {
|
||||
type: 'array',
|
||||
description: 'Domains the credential applies to',
|
||||
items: { type: 'string', description: 'Domain' },
|
||||
},
|
||||
fieldMetadata: {
|
||||
type: 'array',
|
||||
description: 'Fields the credential carries, without their values',
|
||||
items: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
fieldId: { type: 'string', description: 'Field identifier' },
|
||||
label: { type: 'string', description: 'Field name' },
|
||||
type: { type: 'string', description: 'STRING, CONCEALED, or OTP' },
|
||||
},
|
||||
},
|
||||
},
|
||||
hasTotp: { type: 'boolean', description: 'Whether the credential carries a TOTP secret' },
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
@@ -0,0 +1,103 @@
|
||||
import { tinyfishAgentHosting } from '@/tools/tinyfish/hosting'
|
||||
import {
|
||||
RUN_ERROR_OUTPUT_PROPERTIES,
|
||||
SCHEMA_VALIDATION_OUTPUT_PROPERTIES,
|
||||
type TinyFishRawRun,
|
||||
type TinyFishRunParams,
|
||||
type TinyFishRunResponse,
|
||||
} from '@/tools/tinyfish/types'
|
||||
import {
|
||||
AUTOMATION_TOOL_PARAMS,
|
||||
buildAutomationBody,
|
||||
mapRunError,
|
||||
mapSchemaValidation,
|
||||
selectAutomationModelInput,
|
||||
TINYFISH_AGENT_API_BASE,
|
||||
tinyfishErrorMessage,
|
||||
tinyfishHeaders,
|
||||
} from '@/tools/tinyfish/utils'
|
||||
import type { ToolConfig } from '@/tools/types'
|
||||
|
||||
export const runTool: ToolConfig<TinyFishRunParams, TinyFishRunResponse> = {
|
||||
id: 'tinyfish_run',
|
||||
name: 'TinyFish Run Agent',
|
||||
description:
|
||||
'Run a TinyFish web agent against a website and wait for it to finish, returning the structured result it extracted',
|
||||
version: '1.0.0',
|
||||
|
||||
params: { ...AUTOMATION_TOOL_PARAMS },
|
||||
|
||||
hosting: tinyfishAgentHosting(),
|
||||
|
||||
request: {
|
||||
url: `${TINYFISH_AGENT_API_BASE}/v1/automation/run`,
|
||||
method: 'POST',
|
||||
headers: (params) => tinyfishHeaders(params.apiKey),
|
||||
body: (params) => buildAutomationBody(params),
|
||||
modelInput: {
|
||||
mode: 'project',
|
||||
select: selectAutomationModelInput,
|
||||
},
|
||||
},
|
||||
|
||||
transformResponse: async (response: Response) => {
|
||||
if (!response.ok) {
|
||||
throw new Error(await tinyfishErrorMessage(response))
|
||||
}
|
||||
|
||||
const data = (await response.json()) as TinyFishRawRun
|
||||
|
||||
return {
|
||||
/**
|
||||
* A failed run is reported inside a 200 response, so success comes from
|
||||
* the run's own status rather than the HTTP status.
|
||||
*/
|
||||
success: data.status === 'COMPLETED',
|
||||
output: {
|
||||
runId: data.run_id ?? null,
|
||||
status: data.status ?? 'FAILED',
|
||||
startedAt: data.started_at ?? null,
|
||||
finishedAt: data.finished_at ?? null,
|
||||
numOfSteps: data.num_of_steps ?? null,
|
||||
result: data.result ?? null,
|
||||
schemaValidation: mapSchemaValidation(data.schema_validation),
|
||||
error: mapRunError(data.error),
|
||||
},
|
||||
error: data.error?.message ?? undefined,
|
||||
}
|
||||
},
|
||||
|
||||
outputs: {
|
||||
runId: { type: 'string', description: 'Run identifier', optional: true },
|
||||
status: { type: 'string', description: 'Final run status: COMPLETED or FAILED' },
|
||||
startedAt: {
|
||||
type: 'string',
|
||||
description: 'ISO 8601 timestamp when the run started',
|
||||
optional: true,
|
||||
},
|
||||
finishedAt: {
|
||||
type: 'string',
|
||||
description: 'ISO 8601 timestamp when the run finished',
|
||||
optional: true,
|
||||
},
|
||||
numOfSteps: { type: 'number', description: 'Steps the agent took', optional: true },
|
||||
result: {
|
||||
type: 'json',
|
||||
description: 'Structured data the agent extracted, null when the run failed',
|
||||
optional: true,
|
||||
},
|
||||
schemaValidation: {
|
||||
type: 'object',
|
||||
description: 'Validation of the result against the requested output schema',
|
||||
optional: true,
|
||||
properties: SCHEMA_VALIDATION_OUTPUT_PROPERTIES,
|
||||
},
|
||||
error: {
|
||||
type: 'object',
|
||||
description:
|
||||
'Why the run failed, null when it succeeded. Branch on category to decide whether to retry',
|
||||
optional: true,
|
||||
properties: RUN_ERROR_OUTPUT_PROPERTIES,
|
||||
},
|
||||
},
|
||||
}
|
||||
@@ -0,0 +1,72 @@
|
||||
import type {
|
||||
TinyFishRawRunAsync,
|
||||
TinyFishRunAsyncParams,
|
||||
TinyFishRunAsyncResponse,
|
||||
} from '@/tools/tinyfish/types'
|
||||
import {
|
||||
AUTOMATION_TOOL_PARAMS,
|
||||
buildAutomationBody,
|
||||
selectAutomationModelInput,
|
||||
TINYFISH_AGENT_API_BASE,
|
||||
tinyfishErrorMessage,
|
||||
tinyfishHeaders,
|
||||
} from '@/tools/tinyfish/utils'
|
||||
import type { ToolConfig } from '@/tools/types'
|
||||
|
||||
/**
|
||||
* Starts a run without waiting for it.
|
||||
*
|
||||
* This tool has no `hosting` config: the wallet charge lands on steps the run
|
||||
* takes after the request returns, so a hosted key could never be metered
|
||||
* against it. Callers must bring their own TinyFish API key.
|
||||
*/
|
||||
export const runAsyncTool: ToolConfig<TinyFishRunAsyncParams, TinyFishRunAsyncResponse> = {
|
||||
id: 'tinyfish_run_async',
|
||||
name: 'TinyFish Start Agent Run',
|
||||
description:
|
||||
'Queue a TinyFish web agent run and return its run ID immediately, without waiting for the automation to finish',
|
||||
version: '1.0.0',
|
||||
|
||||
params: {
|
||||
...AUTOMATION_TOOL_PARAMS,
|
||||
webhookUrl: {
|
||||
type: 'string',
|
||||
required: false,
|
||||
visibility: 'user-only',
|
||||
description: 'HTTPS URL notified when the run completes, fails, or is cancelled',
|
||||
},
|
||||
},
|
||||
|
||||
request: {
|
||||
url: `${TINYFISH_AGENT_API_BASE}/v1/automation/run-async`,
|
||||
method: 'POST',
|
||||
headers: (params) => tinyfishHeaders(params.apiKey),
|
||||
body: (params) => buildAutomationBody(params),
|
||||
modelInput: {
|
||||
mode: 'project',
|
||||
select: selectAutomationModelInput,
|
||||
},
|
||||
},
|
||||
|
||||
transformResponse: async (response: Response) => {
|
||||
if (!response.ok) {
|
||||
throw new Error(await tinyfishErrorMessage(response))
|
||||
}
|
||||
|
||||
const data = (await response.json()) as TinyFishRawRunAsync
|
||||
|
||||
return {
|
||||
success: !data.error,
|
||||
output: { runId: data.run_id ?? null },
|
||||
error: data.error?.message ?? undefined,
|
||||
}
|
||||
},
|
||||
|
||||
outputs: {
|
||||
runId: {
|
||||
type: 'string',
|
||||
description: 'Identifier of the queued run, used to poll or cancel it',
|
||||
optional: true,
|
||||
},
|
||||
},
|
||||
}
|
||||
@@ -0,0 +1,102 @@
|
||||
import { tinyfishSearchHosting } from '@/tools/tinyfish/hosting'
|
||||
import type {
|
||||
TinyFishRawSearch,
|
||||
TinyFishSearchParams,
|
||||
TinyFishSearchResponse,
|
||||
} from '@/tools/tinyfish/types'
|
||||
import {
|
||||
TINYFISH_SEARCH_API_BASE,
|
||||
tinyfishErrorMessage,
|
||||
tinyfishHeaders,
|
||||
} from '@/tools/tinyfish/utils'
|
||||
import type { ToolConfig } from '@/tools/types'
|
||||
|
||||
export const searchTool: ToolConfig<TinyFishSearchParams, TinyFishSearchResponse> = {
|
||||
id: 'tinyfish_search',
|
||||
name: 'TinyFish Search',
|
||||
description:
|
||||
'Search the web with TinyFish and get ranked results with titles, snippets, and URLs',
|
||||
version: '1.0.0',
|
||||
|
||||
params: {
|
||||
query: {
|
||||
type: 'string',
|
||||
required: true,
|
||||
visibility: 'user-or-llm',
|
||||
description: 'Search query',
|
||||
},
|
||||
location: {
|
||||
type: 'string',
|
||||
required: false,
|
||||
visibility: 'user-or-llm',
|
||||
description: 'Country code for geo-targeted results, such as US',
|
||||
},
|
||||
language: {
|
||||
type: 'string',
|
||||
required: false,
|
||||
visibility: 'user-or-llm',
|
||||
description: 'Language code for the results, such as en',
|
||||
},
|
||||
apiKey: {
|
||||
type: 'string',
|
||||
required: true,
|
||||
visibility: 'user-only',
|
||||
description: 'TinyFish API key',
|
||||
},
|
||||
},
|
||||
|
||||
hosting: tinyfishSearchHosting(),
|
||||
|
||||
request: {
|
||||
url: (params) => {
|
||||
const query = new URLSearchParams({ query: params.query })
|
||||
if (params.location) query.set('location', params.location)
|
||||
if (params.language) query.set('language', params.language)
|
||||
return `${TINYFISH_SEARCH_API_BASE}/?${query.toString()}`
|
||||
},
|
||||
method: 'GET',
|
||||
headers: (params) => tinyfishHeaders(params.apiKey),
|
||||
},
|
||||
|
||||
transformResponse: async (response: Response) => {
|
||||
if (!response.ok) {
|
||||
throw new Error(await tinyfishErrorMessage(response))
|
||||
}
|
||||
|
||||
const data = (await response.json()) as TinyFishRawSearch
|
||||
|
||||
return {
|
||||
success: true,
|
||||
output: {
|
||||
query: data.query ?? '',
|
||||
results: (data.results ?? []).map((result) => ({
|
||||
position: result?.position ?? 0,
|
||||
siteName: result?.site_name ?? '',
|
||||
snippet: result?.snippet ?? '',
|
||||
title: result?.title ?? '',
|
||||
url: result?.url ?? '',
|
||||
})),
|
||||
totalResults: data.total_results ?? 0,
|
||||
},
|
||||
}
|
||||
},
|
||||
|
||||
outputs: {
|
||||
query: { type: 'string', description: 'Query that was executed' },
|
||||
results: {
|
||||
type: 'array',
|
||||
description: 'Ranked search results',
|
||||
items: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
position: { type: 'number', description: 'Rank in the result list' },
|
||||
siteName: { type: 'string', description: 'Site the result came from' },
|
||||
snippet: { type: 'string', description: 'Text snippet from the page' },
|
||||
title: { type: 'string', description: 'Page title' },
|
||||
url: { type: 'string', description: 'Result URL' },
|
||||
},
|
||||
},
|
||||
},
|
||||
totalResults: { type: 'number', description: 'Number of results returned' },
|
||||
},
|
||||
}
|
||||
@@ -0,0 +1,635 @@
|
||||
/**
|
||||
* @vitest-environment node
|
||||
*/
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { TinyFishBlock } from '@/blocks/blocks/tinyfish'
|
||||
import { cancelRunTool } from '@/tools/tinyfish/cancel_run'
|
||||
import { fetchUrlsTool } from '@/tools/tinyfish/fetch_urls'
|
||||
import { getRunTool } from '@/tools/tinyfish/get_run'
|
||||
import { TINYFISH_AGENT_STEP_USD } from '@/tools/tinyfish/hosting'
|
||||
import { listRunsTool } from '@/tools/tinyfish/list_runs'
|
||||
import { listVaultItemsTool } from '@/tools/tinyfish/list_vault_items'
|
||||
import { runTool } from '@/tools/tinyfish/run'
|
||||
import { runAsyncTool } from '@/tools/tinyfish/run_async'
|
||||
import { searchTool } from '@/tools/tinyfish/search'
|
||||
import { buildAutomationBody, parseJsonSchema, parseList } from '@/tools/tinyfish/utils'
|
||||
|
||||
const API_KEY = 'test-key'
|
||||
|
||||
function jsonResponse(body: unknown, init?: ResponseInit) {
|
||||
return new Response(JSON.stringify(body), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
...init,
|
||||
})
|
||||
}
|
||||
|
||||
describe('buildAutomationBody', () => {
|
||||
it('sends only the required fields plus the integration tag by default', () => {
|
||||
expect(buildAutomationBody({ url: 'https://example.com', goal: 'Find pricing' })).toEqual({
|
||||
url: 'https://example.com',
|
||||
goal: 'Find pricing',
|
||||
api_integration: 'sim',
|
||||
})
|
||||
})
|
||||
|
||||
it('nests agent mode and max steps under agent_config', () => {
|
||||
const body = buildAutomationBody({
|
||||
url: 'https://example.com',
|
||||
goal: 'Find pricing',
|
||||
agentMode: 'strict',
|
||||
maxSteps: 50,
|
||||
})
|
||||
expect(body.agent_config).toEqual({ mode: 'strict', max_steps: 50 })
|
||||
})
|
||||
|
||||
it('omits agent_config entirely when neither field is set', () => {
|
||||
const body = buildAutomationBody({ url: 'https://example.com', goal: 'Find pricing' })
|
||||
expect(body).not.toHaveProperty('agent_config')
|
||||
})
|
||||
|
||||
it('sends the Tetra proxy with a country when the proxy is enabled', () => {
|
||||
const body = buildAutomationBody({
|
||||
url: 'https://example.com',
|
||||
goal: 'Find pricing',
|
||||
proxyEnabled: true,
|
||||
proxyCountryCode: 'GB',
|
||||
})
|
||||
expect(body.proxy_config).toEqual({ enabled: true, type: 'tetra', country_code: 'GB' })
|
||||
})
|
||||
|
||||
it('drops a country code when the proxy is off, rather than sending a disabled proxy', () => {
|
||||
const body = buildAutomationBody({
|
||||
url: 'https://example.com',
|
||||
goal: 'Find pricing',
|
||||
proxyCountryCode: 'GB',
|
||||
})
|
||||
expect(body).not.toHaveProperty('proxy_config')
|
||||
})
|
||||
|
||||
it('scopes vault credentials only when the vault is opted into', () => {
|
||||
const scoped = buildAutomationBody({
|
||||
url: 'https://example.com',
|
||||
goal: 'Log in',
|
||||
useVault: true,
|
||||
credentialItemIds: 'cred:a:Work:1, cred:b:Home:2',
|
||||
})
|
||||
expect(scoped.use_vault).toBe(true)
|
||||
expect(scoped.credential_item_ids).toEqual(['cred:a:Work:1', 'cred:b:Home:2'])
|
||||
|
||||
const unscoped = buildAutomationBody({
|
||||
url: 'https://example.com',
|
||||
goal: 'Log in',
|
||||
credentialItemIds: 'cred:a:Work:1',
|
||||
})
|
||||
expect(unscoped).not.toHaveProperty('use_vault')
|
||||
expect(unscoped).not.toHaveProperty('credential_item_ids')
|
||||
})
|
||||
|
||||
it('parses a stringified output schema into an object', () => {
|
||||
const body = buildAutomationBody({
|
||||
url: 'https://example.com',
|
||||
goal: 'Find pricing',
|
||||
outputSchema: '{"type":"object","properties":{"price":{"type":"number"}}}',
|
||||
})
|
||||
expect(body.output_schema).toEqual({
|
||||
type: 'object',
|
||||
properties: { price: { type: 'number' } },
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
describe('parseJsonSchema', () => {
|
||||
it('treats an empty or whitespace-only schema as unset', () => {
|
||||
expect(parseJsonSchema(' ')).toBeUndefined()
|
||||
expect(parseJsonSchema(undefined)).toBeUndefined()
|
||||
})
|
||||
|
||||
it('rejects a schema that is not a JSON object', () => {
|
||||
expect(() => parseJsonSchema('[1, 2]')).toThrow('Output Schema must be a JSON object')
|
||||
})
|
||||
|
||||
it('rejects an already-parsed array, which the executor can hand a json input', () => {
|
||||
expect(() => parseJsonSchema([1, 2] as unknown as Record<string, unknown>)).toThrow(
|
||||
'Output Schema must be a JSON object'
|
||||
)
|
||||
})
|
||||
|
||||
it('names malformed JSON instead of leaking a bare SyntaxError', () => {
|
||||
expect(() => parseJsonSchema('{"type":')).toThrow('Output Schema is not valid JSON')
|
||||
})
|
||||
})
|
||||
|
||||
describe('parseList', () => {
|
||||
it('splits on commas and newlines and drops blank entries', () => {
|
||||
expect(parseList('https://a.com,\n https://b.com , ')).toEqual([
|
||||
'https://a.com',
|
||||
'https://b.com',
|
||||
])
|
||||
})
|
||||
|
||||
it('passes an array through untouched apart from trimming', () => {
|
||||
expect(parseList([' https://a.com ', ''])).toEqual(['https://a.com'])
|
||||
})
|
||||
})
|
||||
|
||||
describe('tinyfish_run', () => {
|
||||
it('maps a completed run onto camelCase outputs', async () => {
|
||||
const result = await runTool.transformResponse!(
|
||||
jsonResponse({
|
||||
run_id: 'run-1',
|
||||
status: 'COMPLETED',
|
||||
started_at: '2026-01-01T00:00:00Z',
|
||||
finished_at: '2026-01-01T00:00:30Z',
|
||||
num_of_steps: 5,
|
||||
result: { price: 799 },
|
||||
schema_validation: { valid: true, re_prompt_attempts: 0, errors: [] },
|
||||
error: null,
|
||||
})
|
||||
)
|
||||
|
||||
expect(result.success).toBe(true)
|
||||
expect(result.output).toEqual({
|
||||
runId: 'run-1',
|
||||
status: 'COMPLETED',
|
||||
startedAt: '2026-01-01T00:00:00Z',
|
||||
finishedAt: '2026-01-01T00:00:30Z',
|
||||
numOfSteps: 5,
|
||||
result: { price: 799 },
|
||||
schemaValidation: { valid: true, rePromptAttempts: 0, errors: [] },
|
||||
error: null,
|
||||
})
|
||||
})
|
||||
|
||||
it('reports a FAILED run as a failure even though the HTTP status is 200', async () => {
|
||||
const result = await runTool.transformResponse!(
|
||||
jsonResponse({
|
||||
run_id: 'run-2',
|
||||
status: 'FAILED',
|
||||
started_at: '2026-01-01T00:00:00Z',
|
||||
finished_at: '2026-01-01T00:02:00Z',
|
||||
num_of_steps: 42,
|
||||
result: null,
|
||||
schema_validation: null,
|
||||
error: {
|
||||
code: 'service_busy',
|
||||
message: 'Browser crashed',
|
||||
category: 'SYSTEM_FAILURE',
|
||||
retry_after: 60,
|
||||
help_url: 'https://docs.tinyfish.ai/prompting-guide',
|
||||
},
|
||||
})
|
||||
)
|
||||
|
||||
expect(result.success).toBe(false)
|
||||
expect(result.error).toBe('Browser crashed')
|
||||
expect(result.output.result).toBeNull()
|
||||
})
|
||||
|
||||
it('exposes the error category and retry delay so a workflow can branch on them', async () => {
|
||||
const result = await runTool.transformResponse!(
|
||||
jsonResponse({
|
||||
run_id: 'run-2',
|
||||
status: 'FAILED',
|
||||
started_at: null,
|
||||
finished_at: null,
|
||||
num_of_steps: 42,
|
||||
result: null,
|
||||
schema_validation: null,
|
||||
error: {
|
||||
code: 'service_busy',
|
||||
message: 'Browser crashed',
|
||||
category: 'SYSTEM_FAILURE',
|
||||
retry_after: 60,
|
||||
help_url: 'https://docs.tinyfish.ai/prompting-guide',
|
||||
},
|
||||
})
|
||||
)
|
||||
|
||||
expect(result.output.error).toEqual({
|
||||
code: 'service_busy',
|
||||
message: 'Browser crashed',
|
||||
category: 'SYSTEM_FAILURE',
|
||||
retryAfter: 60,
|
||||
helpUrl: 'https://docs.tinyfish.ai/prompting-guide',
|
||||
helpMessage: null,
|
||||
})
|
||||
})
|
||||
|
||||
/**
|
||||
* Documents a known gap rather than asserting desired behavior: the executor
|
||||
* meters only successful executions, so the steps a failed run consumed are
|
||||
* charged to the hosted wallet and billed to nobody.
|
||||
*/
|
||||
it('still reports the step count of a failed run, which goes unbilled today', async () => {
|
||||
const result = await runTool.transformResponse!(
|
||||
jsonResponse({
|
||||
run_id: 'run-2',
|
||||
status: 'FAILED',
|
||||
started_at: null,
|
||||
finished_at: null,
|
||||
num_of_steps: 42,
|
||||
result: null,
|
||||
schema_validation: null,
|
||||
error: { message: 'Browser crashed', category: 'SYSTEM_FAILURE' },
|
||||
})
|
||||
)
|
||||
|
||||
expect(result.success).toBe(false)
|
||||
expect(result.output.numOfSteps).toBe(42)
|
||||
})
|
||||
|
||||
it('surfaces the API error code and message on a non-2xx response', async () => {
|
||||
await expect(
|
||||
runTool.transformResponse!(
|
||||
jsonResponse(
|
||||
{ error: { code: 'INVALID_API_KEY', message: 'Invalid or expired API key' } },
|
||||
{ status: 401 }
|
||||
)
|
||||
)
|
||||
).rejects.toThrow('INVALID_API_KEY: Invalid or expired API key')
|
||||
})
|
||||
|
||||
it('falls back to the status line when the error body is not parseable', async () => {
|
||||
await expect(
|
||||
runTool.transformResponse!(new Response('gateway down', { status: 502 }))
|
||||
).rejects.toThrow('TinyFish request failed with status 502')
|
||||
})
|
||||
})
|
||||
|
||||
describe('tinyfish model input', () => {
|
||||
it('projects the goal, which the agent model reads verbatim', () => {
|
||||
const select =
|
||||
runTool.request.modelInput?.mode === 'project' && runTool.request.modelInput.select
|
||||
expect(select).toBeTruthy()
|
||||
expect(
|
||||
(select as (p: never) => unknown)({
|
||||
goal: 'Find pricing',
|
||||
outputSchema: '{"type":"object"}',
|
||||
} as never)
|
||||
).toEqual({ goal: 'Find pricing', outputSchema: '{"type":"object"}' })
|
||||
})
|
||||
|
||||
it('leaves the target URL unprojected, since it is a resource locator', () => {
|
||||
const select =
|
||||
runTool.request.modelInput?.mode === 'project' && runTool.request.modelInput.select
|
||||
const selection = (select as (p: never) => Record<string, unknown>)({
|
||||
goal: 'Find pricing',
|
||||
url: 'https://example.com',
|
||||
} as never)
|
||||
expect(selection).not.toHaveProperty('url')
|
||||
})
|
||||
|
||||
it('projects the output schema, which TinyFish re-prompts the agent model with', () => {
|
||||
const select =
|
||||
runTool.request.modelInput?.mode === 'project' && runTool.request.modelInput.select
|
||||
const selection = (select as (p: never) => Record<string, unknown>)({
|
||||
goal: 'Find pricing',
|
||||
outputSchema: { type: 'object' },
|
||||
} as never)
|
||||
expect(selection.outputSchema).toEqual({ type: 'object' })
|
||||
})
|
||||
|
||||
it('projects the goal on the async run too', () => {
|
||||
expect(runAsyncTool.request.modelInput?.mode).toBe('project')
|
||||
})
|
||||
})
|
||||
|
||||
describe('tinyfish hosted-key config', () => {
|
||||
it('bills the synchronous run on the steps the API reported', () => {
|
||||
const pricing = runTool.hosting?.pricing
|
||||
if (pricing?.type !== 'custom') throw new Error('expected custom pricing')
|
||||
|
||||
expect(pricing.getCost({} as never, { numOfSteps: 5 })).toEqual({
|
||||
cost: 5 * TINYFISH_AGENT_STEP_USD,
|
||||
metadata: { steps: 5 },
|
||||
})
|
||||
})
|
||||
|
||||
it('refuses to bill a run whose step count is missing or unusable', () => {
|
||||
const pricing = runTool.hosting?.pricing
|
||||
if (pricing?.type !== 'custom') throw new Error('expected custom pricing')
|
||||
|
||||
expect(() => pricing.getCost({} as never, { numOfSteps: null })).toThrow('num_of_steps')
|
||||
expect(() => pricing.getCost({} as never, { numOfSteps: 'five' })).toThrow('non-numeric')
|
||||
})
|
||||
|
||||
it('charges nothing for the free Search and Fetch products', () => {
|
||||
expect(searchTool.hosting?.pricing).toEqual({ type: 'per_request', cost: 0 })
|
||||
expect(fetchUrlsTool.hosting?.pricing).toEqual({ type: 'per_request', cost: 0 })
|
||||
})
|
||||
|
||||
it('throttles Fetch on URLs, the axis TinyFish itself limits', () => {
|
||||
const rateLimit = fetchUrlsTool.hosting?.rateLimit
|
||||
if (rateLimit?.mode !== 'custom') throw new Error('expected custom rate limit')
|
||||
|
||||
const urls = rateLimit.dimensions.find((dimension) => dimension.name === 'urls')
|
||||
expect(urls?.extractUsage({ urls: 'https://a.com, https://b.com' }, {})).toBe(2)
|
||||
expect(urls?.extractUsage({}, {})).toBe(0)
|
||||
})
|
||||
|
||||
it('leaves the async run and its companions off hosted keys, since they cannot be metered', () => {
|
||||
expect(runAsyncTool.hosting).toBeUndefined()
|
||||
expect(getRunTool.hosting).toBeUndefined()
|
||||
expect(cancelRunTool.hosting).toBeUndefined()
|
||||
expect(listRunsTool.hosting).toBeUndefined()
|
||||
expect(listVaultItemsTool.hosting).toBeUndefined()
|
||||
})
|
||||
})
|
||||
|
||||
describe('tinyfish_get_run', () => {
|
||||
it('maps the run summary, recording, and steps', async () => {
|
||||
const result = await getRunTool.transformResponse!(
|
||||
jsonResponse({
|
||||
run_id: 'run-1',
|
||||
status: 'RUNNING',
|
||||
goal: 'Find pricing',
|
||||
created_at: '2026-01-01T00:00:00Z',
|
||||
started_at: '2026-01-01T00:00:05Z',
|
||||
finished_at: null,
|
||||
num_of_steps: null,
|
||||
result: null,
|
||||
schema_validation: null,
|
||||
error: null,
|
||||
streaming_url: 'https://stream.agent.tinyfish.ai/session/xyz',
|
||||
browser_config: { proxy_enabled: true, proxy_country_code: 'US' },
|
||||
video_url: null,
|
||||
steps: [
|
||||
{
|
||||
id: 'evt_1',
|
||||
timestamp: '2026-01-01T00:00:06Z',
|
||||
status: 'RUNNING',
|
||||
action: 'Click the pricing link',
|
||||
screenshot: null,
|
||||
duration: '1.2s',
|
||||
},
|
||||
],
|
||||
})
|
||||
)
|
||||
|
||||
expect(result.success).toBe(true)
|
||||
expect(result.output.streamingUrl).toBe('https://stream.agent.tinyfish.ai/session/xyz')
|
||||
expect(result.output.browserConfig).toEqual({ proxyEnabled: true, proxyCountryCode: 'US' })
|
||||
expect(result.output.numOfSteps).toBeNull()
|
||||
expect(result.output.steps).toEqual([
|
||||
{
|
||||
id: 'evt_1',
|
||||
timestamp: '2026-01-01T00:00:06Z',
|
||||
status: 'RUNNING',
|
||||
action: 'Click the pricing link',
|
||||
duration: '1.2s',
|
||||
},
|
||||
])
|
||||
})
|
||||
|
||||
it('percent-encodes the run id into the path', () => {
|
||||
const url = getRunTool.request.url as (params: { runId: string }) => string
|
||||
expect(url({ runId: ' a/b ' })).toBe('https://agent.tinyfish.ai/v1/runs/a%2Fb')
|
||||
})
|
||||
})
|
||||
|
||||
describe('tinyfish_cancel_run', () => {
|
||||
it('reports an already-finished run without claiming it was cancelled', async () => {
|
||||
const result = await cancelRunTool.transformResponse!(
|
||||
jsonResponse({
|
||||
run_id: 'run-1',
|
||||
status: 'COMPLETED',
|
||||
cancelled_at: null,
|
||||
message: 'Run already finished',
|
||||
})
|
||||
)
|
||||
|
||||
expect(result.output).toEqual({
|
||||
runId: 'run-1',
|
||||
status: 'COMPLETED',
|
||||
cancelledAt: null,
|
||||
message: 'Run already finished',
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
describe('tinyfish_list_runs', () => {
|
||||
it('only sends the filters that were set', () => {
|
||||
const url = listRunsTool.request.url as (params: Record<string, unknown>) => string
|
||||
expect(url({ apiKey: API_KEY })).toBe('https://agent.tinyfish.ai/v1/runs')
|
||||
expect(url({ apiKey: API_KEY, status: 'FAILED', limit: 50 })).toBe(
|
||||
'https://agent.tinyfish.ai/v1/runs?status=FAILED&limit=50'
|
||||
)
|
||||
})
|
||||
|
||||
it('maps the paginated envelope', async () => {
|
||||
const result = await listRunsTool.transformResponse!(
|
||||
jsonResponse({
|
||||
data: [
|
||||
{
|
||||
run_id: 'run-1',
|
||||
status: 'COMPLETED',
|
||||
goal: 'Find pricing',
|
||||
created_at: '2026-01-01T00:00:00Z',
|
||||
started_at: null,
|
||||
finished_at: null,
|
||||
num_of_steps: 3,
|
||||
result: null,
|
||||
schema_validation: null,
|
||||
error: null,
|
||||
streaming_url: null,
|
||||
browser_config: null,
|
||||
},
|
||||
],
|
||||
pagination: { total: 42, next_cursor: 'cursor-2', has_more: true },
|
||||
})
|
||||
)
|
||||
|
||||
expect(result.output.runs).toHaveLength(1)
|
||||
expect(result.output.runs[0].runId).toBe('run-1')
|
||||
expect(result.output).toMatchObject({ total: 42, nextCursor: 'cursor-2', hasMore: true })
|
||||
})
|
||||
})
|
||||
|
||||
describe('tinyfish_search', () => {
|
||||
it('builds the query string from the query and optional locale filters', () => {
|
||||
const url = searchTool.request.url as (params: Record<string, unknown>) => string
|
||||
expect(url({ query: 'web automation', apiKey: API_KEY })).toBe(
|
||||
'https://api.search.tinyfish.ai/?query=web+automation'
|
||||
)
|
||||
expect(url({ query: 'news', location: 'US', language: 'en', apiKey: API_KEY })).toBe(
|
||||
'https://api.search.tinyfish.ai/?query=news&location=US&language=en'
|
||||
)
|
||||
})
|
||||
|
||||
it('maps ranked results onto camelCase fields', async () => {
|
||||
const result = await searchTool.transformResponse!(
|
||||
jsonResponse({
|
||||
query: 'web automation',
|
||||
results: [
|
||||
{
|
||||
position: 1,
|
||||
site_name: 'example.com',
|
||||
snippet: 'Top tools',
|
||||
title: 'Best Tools',
|
||||
url: 'https://example.com/tools',
|
||||
},
|
||||
],
|
||||
total_results: 1,
|
||||
})
|
||||
)
|
||||
|
||||
expect(result.output.results[0]).toEqual({
|
||||
position: 1,
|
||||
siteName: 'example.com',
|
||||
snippet: 'Top tools',
|
||||
title: 'Best Tools',
|
||||
url: 'https://example.com/tools',
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
describe('tinyfish_fetch', () => {
|
||||
it('splits the URL list and sends extraction flags only when enabled', () => {
|
||||
const body = fetchUrlsTool.request.body!({
|
||||
urls: 'https://a.com, https://b.com',
|
||||
format: 'html',
|
||||
links: true,
|
||||
apiKey: API_KEY,
|
||||
} as never) as Record<string, unknown>
|
||||
|
||||
expect(body).toEqual({
|
||||
urls: ['https://a.com', 'https://b.com'],
|
||||
format: 'html',
|
||||
links: true,
|
||||
})
|
||||
})
|
||||
|
||||
it('keeps per-URL failures alongside the successful results', async () => {
|
||||
const result = await fetchUrlsTool.transformResponse!(
|
||||
jsonResponse({
|
||||
results: [
|
||||
{
|
||||
url: 'https://a.com',
|
||||
final_url: 'https://www.a.com',
|
||||
title: 'A',
|
||||
description: null,
|
||||
language: 'en',
|
||||
format: 'markdown',
|
||||
text: '# A',
|
||||
author: null,
|
||||
published_date: null,
|
||||
latency_ms: 120,
|
||||
},
|
||||
],
|
||||
errors: [{ url: 'https://b.com', error: 'Failed to fetch resource' }],
|
||||
})
|
||||
)
|
||||
|
||||
expect(result.success).toBe(true)
|
||||
expect(result.output.results[0]).toMatchObject({
|
||||
finalUrl: 'https://www.a.com',
|
||||
publishedDate: null,
|
||||
latencyMs: 120,
|
||||
links: [],
|
||||
imageLinks: [],
|
||||
})
|
||||
expect(result.output.errors).toEqual([
|
||||
{ url: 'https://b.com', error: 'Failed to fetch resource' },
|
||||
])
|
||||
})
|
||||
})
|
||||
|
||||
describe('tinyfish_fetch bounds', () => {
|
||||
function body(urls: string) {
|
||||
return () => fetchUrlsTool.request.body!({ urls, apiKey: API_KEY } as never)
|
||||
}
|
||||
|
||||
it('names the empty and over-long cases instead of letting the API 400', () => {
|
||||
expect(body(' , ,')).toThrow('At least one URL is required')
|
||||
expect(
|
||||
body(Array.from({ length: 11 }, (_, index) => `https://a${index}.com`).join(','))
|
||||
).toThrow('at most 10 URLs')
|
||||
})
|
||||
|
||||
it('accepts exactly the documented maximum', () => {
|
||||
expect(
|
||||
body(Array.from({ length: 10 }, (_, index) => `https://a${index}.com`).join(','))
|
||||
).not.toThrow()
|
||||
})
|
||||
})
|
||||
|
||||
describe('tinyfish_list_vault_items', () => {
|
||||
it('returns display-safe metadata with the URIs a run scopes itself to', async () => {
|
||||
const result = await listVaultItemsTool.transformResponse!(
|
||||
jsonResponse({
|
||||
items: [
|
||||
{
|
||||
itemId: 'cred:conn-123:Personal:item-abc',
|
||||
connectionId: 'conn_123',
|
||||
label: 'Amazon Login',
|
||||
vaultName: 'Personal',
|
||||
domains: ['amazon.com'],
|
||||
fieldMetadata: [{ fieldId: 'password', label: 'Password', type: 'CONCEALED' }],
|
||||
hasTotp: true,
|
||||
},
|
||||
],
|
||||
})
|
||||
)
|
||||
|
||||
expect(result.output.items[0]).toEqual({
|
||||
itemId: 'cred:conn-123:Personal:item-abc',
|
||||
connectionId: 'conn_123',
|
||||
label: 'Amazon Login',
|
||||
vaultName: 'Personal',
|
||||
domains: ['amazon.com'],
|
||||
fieldMetadata: [{ fieldId: 'password', label: 'Password', type: 'CONCEALED' }],
|
||||
hasTotp: true,
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
describe('TinyFish block', () => {
|
||||
it('routes every operation to its own tool', () => {
|
||||
for (const toolId of TinyFishBlock.tools.access) {
|
||||
expect(TinyFishBlock.tools.config?.tool?.({ operation: toolId })).toBe(toolId)
|
||||
}
|
||||
})
|
||||
|
||||
it('falls back to the synchronous run for an unknown operation', () => {
|
||||
expect(TinyFishBlock.tools.config?.tool?.({ operation: 'nope' })).toBe('tinyfish_run')
|
||||
})
|
||||
|
||||
it('renames the list-runs goal filter onto the goal query the tool sends', () => {
|
||||
const params = TinyFishBlock.tools.config?.params?.({
|
||||
operation: 'tinyfish_list_runs',
|
||||
goalFilter: 'pricing',
|
||||
})
|
||||
expect(params).toMatchObject({ goal: 'pricing' })
|
||||
})
|
||||
|
||||
it('leaves the automation goal alone on a run operation', () => {
|
||||
const params = TinyFishBlock.tools.config?.params?.({
|
||||
operation: 'tinyfish_run',
|
||||
goalFilter: 'pricing',
|
||||
})
|
||||
expect(params).not.toHaveProperty('goal')
|
||||
})
|
||||
|
||||
it('coerces the numeric text inputs and drops them when blank', () => {
|
||||
expect(
|
||||
TinyFishBlock.tools.config?.params?.({ operation: 'tinyfish_run', maxSteps: '50' })
|
||||
).toMatchObject({ maxSteps: 50 })
|
||||
expect(
|
||||
TinyFishBlock.tools.config?.params?.({ operation: 'tinyfish_run', maxSteps: ' ' })
|
||||
).not.toHaveProperty('maxSteps')
|
||||
})
|
||||
|
||||
it('always shows an API key field for the operations hosted keys cannot cover', () => {
|
||||
const apiKeyFields = TinyFishBlock.subBlocks.filter((subBlock) => subBlock.id === 'apiKey')
|
||||
expect(apiKeyFields).toHaveLength(2)
|
||||
|
||||
const hosted = apiKeyFields.find((field) => field.hideWhenHosted)
|
||||
const unhosted = apiKeyFields.find((field) => !field.hideWhenHosted)
|
||||
expect(hosted?.condition).toMatchObject({
|
||||
field: 'operation',
|
||||
value: ['tinyfish_run', 'tinyfish_search', 'tinyfish_fetch'],
|
||||
})
|
||||
expect(unhosted?.condition).toMatchObject({ not: true })
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,482 @@
|
||||
import type { ToolResponse } from '@/tools/types'
|
||||
|
||||
/** Browser engine TinyFish runs the agent in. */
|
||||
export type TinyFishBrowserProfile = 'lite' | 'stealth'
|
||||
|
||||
/** Agent behavior mode. `strict` fails fast, which suits test automation. */
|
||||
export type TinyFishAgentMode = 'default' | 'strict'
|
||||
|
||||
/** Lifecycle states a TinyFish run can be in. */
|
||||
export type TinyFishRunStatus = 'PENDING' | 'RUNNING' | 'COMPLETED' | 'FAILED' | 'CANCELLED'
|
||||
|
||||
/** Classification TinyFish attaches to a failed run. */
|
||||
export type TinyFishErrorCategory =
|
||||
| 'SYSTEM_FAILURE'
|
||||
| 'AGENT_FAILURE'
|
||||
| 'BILLING_FAILURE'
|
||||
| 'UNKNOWN'
|
||||
|
||||
/** Output format the Fetch API extracts page content into. */
|
||||
export type TinyFishFetchFormat = 'markdown' | 'html' | 'json'
|
||||
|
||||
interface TinyFishApiKeyParams {
|
||||
apiKey: string
|
||||
}
|
||||
|
||||
/** Request fields shared by the synchronous and asynchronous automation endpoints. */
|
||||
interface TinyFishAutomationParams extends TinyFishApiKeyParams {
|
||||
url: string
|
||||
goal: string
|
||||
browserProfile?: TinyFishBrowserProfile
|
||||
agentMode?: TinyFishAgentMode
|
||||
maxSteps?: number
|
||||
outputSchema?: string | Record<string, unknown>
|
||||
proxyEnabled?: boolean
|
||||
proxyCountryCode?: string
|
||||
useVault?: boolean
|
||||
credentialItemIds?: string | string[]
|
||||
}
|
||||
|
||||
export interface TinyFishRunParams extends TinyFishAutomationParams {}
|
||||
|
||||
export interface TinyFishRunAsyncParams extends TinyFishAutomationParams {
|
||||
webhookUrl?: string
|
||||
}
|
||||
|
||||
export interface TinyFishGetRunParams extends TinyFishApiKeyParams {
|
||||
runId: string
|
||||
}
|
||||
|
||||
export interface TinyFishCancelRunParams extends TinyFishApiKeyParams {
|
||||
runId: string
|
||||
}
|
||||
|
||||
export interface TinyFishListRunsParams extends TinyFishApiKeyParams {
|
||||
status?: TinyFishRunStatus
|
||||
goal?: string
|
||||
createdAfter?: string
|
||||
createdBefore?: string
|
||||
sortDirection?: 'asc' | 'desc'
|
||||
cursor?: string
|
||||
limit?: number
|
||||
}
|
||||
|
||||
export interface TinyFishListVaultItemsParams extends TinyFishApiKeyParams {}
|
||||
|
||||
export interface TinyFishSearchParams extends TinyFishApiKeyParams {
|
||||
query: string
|
||||
location?: string
|
||||
language?: string
|
||||
}
|
||||
|
||||
export interface TinyFishFetchParams extends TinyFishApiKeyParams {
|
||||
urls: string | string[]
|
||||
format?: TinyFishFetchFormat
|
||||
links?: boolean
|
||||
imageLinks?: boolean
|
||||
}
|
||||
|
||||
/** A single mismatch between a run result and the requested `output_schema`. */
|
||||
interface TinyFishSchemaValidationError {
|
||||
path: string
|
||||
expected: string
|
||||
received: string
|
||||
message: string
|
||||
}
|
||||
|
||||
/** Outcome of validating a run result against the requested `output_schema`. */
|
||||
export interface TinyFishSchemaValidation {
|
||||
valid: boolean
|
||||
rePromptAttempts: number
|
||||
errors: TinyFishSchemaValidationError[]
|
||||
}
|
||||
|
||||
/** Failure details TinyFish returns inside a 200 response for a failed run. */
|
||||
export interface TinyFishRunError {
|
||||
code: string | null
|
||||
message: string
|
||||
category: TinyFishErrorCategory
|
||||
retryAfter: number | null
|
||||
helpUrl: string | null
|
||||
helpMessage: string | null
|
||||
}
|
||||
|
||||
/** Proxy settings the run actually executed with. */
|
||||
export interface TinyFishBrowserConfig {
|
||||
proxyEnabled: boolean | null
|
||||
proxyCountryCode: string | null
|
||||
}
|
||||
|
||||
/** Run summary shared by the get-run and list-runs endpoints. */
|
||||
export interface TinyFishRunSummary {
|
||||
runId: string
|
||||
status: TinyFishRunStatus
|
||||
goal: string
|
||||
createdAt: string
|
||||
startedAt: string | null
|
||||
finishedAt: string | null
|
||||
numOfSteps: number | null
|
||||
result: Record<string, unknown> | null
|
||||
schemaValidation: TinyFishSchemaValidation | null
|
||||
error: TinyFishRunError | null
|
||||
streamingUrl: string | null
|
||||
browserConfig: TinyFishBrowserConfig | null
|
||||
}
|
||||
|
||||
export interface TinyFishRunResponse extends ToolResponse {
|
||||
output: {
|
||||
runId: string | null
|
||||
status: 'COMPLETED' | 'FAILED'
|
||||
startedAt: string | null
|
||||
finishedAt: string | null
|
||||
numOfSteps: number | null
|
||||
result: Record<string, unknown> | null
|
||||
schemaValidation: TinyFishSchemaValidation | null
|
||||
error: TinyFishRunError | null
|
||||
}
|
||||
}
|
||||
|
||||
export interface TinyFishRunAsyncResponse extends ToolResponse {
|
||||
output: {
|
||||
runId: string | null
|
||||
}
|
||||
}
|
||||
|
||||
/** A single step the agent took during a run. */
|
||||
interface TinyFishRunStep {
|
||||
id: string
|
||||
timestamp: string
|
||||
status: TinyFishRunStatus
|
||||
action: string | null
|
||||
duration: string | null
|
||||
}
|
||||
|
||||
export interface TinyFishGetRunResponse extends ToolResponse {
|
||||
output: TinyFishRunSummary & {
|
||||
videoUrl: string | null
|
||||
steps: TinyFishRunStep[]
|
||||
}
|
||||
}
|
||||
|
||||
export interface TinyFishCancelRunResponse extends ToolResponse {
|
||||
output: {
|
||||
runId: string
|
||||
status: 'CANCELLED' | 'COMPLETED' | 'FAILED'
|
||||
cancelledAt: string | null
|
||||
message: string | null
|
||||
}
|
||||
}
|
||||
|
||||
export interface TinyFishListRunsResponse extends ToolResponse {
|
||||
output: {
|
||||
runs: TinyFishRunSummary[]
|
||||
total: number
|
||||
nextCursor: string | null
|
||||
hasMore: boolean
|
||||
}
|
||||
}
|
||||
|
||||
/** Display-safe metadata for one field of a vault item. */
|
||||
interface TinyFishVaultFieldMetadata {
|
||||
fieldId: string
|
||||
label: string
|
||||
type: 'STRING' | 'CONCEALED' | 'OTP'
|
||||
}
|
||||
|
||||
/** Display-safe metadata for one credential in a connected password manager. */
|
||||
interface TinyFishVaultItem {
|
||||
itemId: string
|
||||
connectionId: string | null
|
||||
label: string
|
||||
vaultName: string
|
||||
domains: string[]
|
||||
fieldMetadata: TinyFishVaultFieldMetadata[]
|
||||
hasTotp: boolean
|
||||
}
|
||||
|
||||
export interface TinyFishListVaultItemsResponse extends ToolResponse {
|
||||
output: {
|
||||
items: TinyFishVaultItem[]
|
||||
}
|
||||
}
|
||||
|
||||
/** A single ranked result from the Search API. */
|
||||
interface TinyFishSearchResult {
|
||||
position: number
|
||||
siteName: string
|
||||
snippet: string
|
||||
title: string
|
||||
url: string
|
||||
}
|
||||
|
||||
export interface TinyFishSearchResponse extends ToolResponse {
|
||||
output: {
|
||||
query: string
|
||||
results: TinyFishSearchResult[]
|
||||
totalResults: number
|
||||
}
|
||||
}
|
||||
|
||||
/** A successfully fetched page from the Fetch API. */
|
||||
interface TinyFishFetchResult {
|
||||
url: string
|
||||
finalUrl: string | null
|
||||
title: string | null
|
||||
description: string | null
|
||||
language: string | null
|
||||
format: TinyFishFetchFormat
|
||||
text: string | Record<string, unknown> | null
|
||||
author: string | null
|
||||
publishedDate: string | null
|
||||
links: string[]
|
||||
imageLinks: string[]
|
||||
latencyMs: number | null
|
||||
}
|
||||
|
||||
/** A URL the Fetch API could not retrieve. Reported per URL, never fatal. */
|
||||
interface TinyFishFetchError {
|
||||
url: string
|
||||
error: string
|
||||
}
|
||||
|
||||
export interface TinyFishFetchResponse extends ToolResponse {
|
||||
output: {
|
||||
results: TinyFishFetchResult[]
|
||||
errors: TinyFishFetchError[]
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Raw snake_case shapes TinyFish returns on the wire.
|
||||
*
|
||||
* Every field is optional and nullable: these describe what a response may
|
||||
* legally omit, so the mapping helpers narrow them into the camelCase output
|
||||
* types above rather than trusting the payload.
|
||||
*/
|
||||
interface TinyFishRawSchemaValidation {
|
||||
valid?: boolean | null
|
||||
re_prompt_attempts?: number | null
|
||||
errors?: Array<{
|
||||
path?: string | null
|
||||
expected?: string | null
|
||||
received?: string | null
|
||||
message?: string | null
|
||||
}> | null
|
||||
}
|
||||
|
||||
interface TinyFishRawRunError {
|
||||
code?: string | null
|
||||
message?: string | null
|
||||
category?: TinyFishErrorCategory | null
|
||||
retry_after?: number | null
|
||||
help_url?: string | null
|
||||
help_message?: string | null
|
||||
}
|
||||
|
||||
interface TinyFishRawBrowserConfig {
|
||||
proxy_enabled?: boolean | null
|
||||
proxy_country_code?: string | null
|
||||
}
|
||||
|
||||
export interface TinyFishRawRunSummary {
|
||||
run_id?: string | null
|
||||
status?: TinyFishRunStatus | null
|
||||
goal?: string | null
|
||||
created_at?: string | null
|
||||
started_at?: string | null
|
||||
finished_at?: string | null
|
||||
num_of_steps?: number | null
|
||||
result?: Record<string, unknown> | null
|
||||
schema_validation?: TinyFishRawSchemaValidation | null
|
||||
error?: TinyFishRawRunError | null
|
||||
streaming_url?: string | null
|
||||
browser_config?: TinyFishRawBrowserConfig | null
|
||||
}
|
||||
|
||||
export interface TinyFishRawRunDetail extends TinyFishRawRunSummary {
|
||||
video_url?: string | null
|
||||
steps?: Array<{
|
||||
id?: string | null
|
||||
timestamp?: string | null
|
||||
status?: TinyFishRunStatus | null
|
||||
action?: string | null
|
||||
duration?: string | null
|
||||
}> | null
|
||||
}
|
||||
|
||||
export interface TinyFishRawRun {
|
||||
run_id?: string | null
|
||||
status?: 'COMPLETED' | 'FAILED'
|
||||
started_at?: string | null
|
||||
finished_at?: string | null
|
||||
num_of_steps?: number | null
|
||||
result?: Record<string, unknown> | null
|
||||
schema_validation?: TinyFishRawSchemaValidation | null
|
||||
error?: TinyFishRawRunError | null
|
||||
}
|
||||
|
||||
export interface TinyFishRawRunAsync {
|
||||
run_id?: string | null
|
||||
error?: { code?: string | null; message?: string | null } | null
|
||||
}
|
||||
|
||||
export interface TinyFishRawCancel {
|
||||
run_id?: string | null
|
||||
status?: 'CANCELLED' | 'COMPLETED' | 'FAILED'
|
||||
cancelled_at?: string | null
|
||||
message?: string | null
|
||||
}
|
||||
|
||||
export interface TinyFishRawRunList {
|
||||
data?: TinyFishRawRunSummary[] | null
|
||||
pagination?: {
|
||||
total?: number | null
|
||||
next_cursor?: string | null
|
||||
has_more?: boolean | null
|
||||
} | null
|
||||
}
|
||||
|
||||
export interface TinyFishRawSearch {
|
||||
query?: string | null
|
||||
results?: Array<{
|
||||
position?: number | null
|
||||
site_name?: string | null
|
||||
snippet?: string | null
|
||||
title?: string | null
|
||||
url?: string | null
|
||||
}> | null
|
||||
total_results?: number | null
|
||||
}
|
||||
|
||||
export interface TinyFishRawFetch {
|
||||
results?: Array<{
|
||||
url?: string | null
|
||||
final_url?: string | null
|
||||
title?: string | null
|
||||
description?: string | null
|
||||
language?: string | null
|
||||
format?: TinyFishFetchFormat | null
|
||||
text?: string | Record<string, unknown> | null
|
||||
author?: string | null
|
||||
published_date?: string | null
|
||||
links?: string[] | null
|
||||
image_links?: string[] | null
|
||||
latency_ms?: number | null
|
||||
}> | null
|
||||
errors?: Array<{ url?: string | null; error?: string | null }> | null
|
||||
}
|
||||
|
||||
export interface TinyFishRawVaultItems {
|
||||
items?: Array<{
|
||||
itemId?: string | null
|
||||
connectionId?: string | null
|
||||
label?: string | null
|
||||
vaultName?: string | null
|
||||
domains?: string[] | null
|
||||
fieldMetadata?: Array<{
|
||||
fieldId?: string | null
|
||||
label?: string | null
|
||||
type?: 'STRING' | 'CONCEALED' | 'OTP' | null
|
||||
}> | null
|
||||
hasTotp?: boolean | null
|
||||
}> | null
|
||||
}
|
||||
|
||||
export type { TinyFishRawBrowserConfig, TinyFishRawRunError, TinyFishRawSchemaValidation }
|
||||
|
||||
/** Output property descriptions for the run-error object, shared by run-returning tools. */
|
||||
export const RUN_ERROR_OUTPUT_PROPERTIES = {
|
||||
code: { type: 'string', description: 'Machine-readable error code', optional: true },
|
||||
message: { type: 'string', description: 'Why the run failed' },
|
||||
category: {
|
||||
type: 'string',
|
||||
description:
|
||||
'SYSTEM_FAILURE (retry), AGENT_FAILURE (fix the goal), BILLING_FAILURE (add credits), or UNKNOWN',
|
||||
},
|
||||
retryAfter: {
|
||||
type: 'number',
|
||||
description: 'Suggested retry delay in seconds, null when not retryable',
|
||||
optional: true,
|
||||
},
|
||||
helpUrl: { type: 'string', description: 'Troubleshooting documentation URL', optional: true },
|
||||
helpMessage: { type: 'string', description: 'Human-readable guidance', optional: true },
|
||||
} as const
|
||||
|
||||
/** Output property descriptions for the schema-validation object. */
|
||||
export const SCHEMA_VALIDATION_OUTPUT_PROPERTIES = {
|
||||
valid: { type: 'boolean', description: 'Whether the result matched the requested output schema' },
|
||||
rePromptAttempts: {
|
||||
type: 'number',
|
||||
description: 'Number of schema-repair re-prompts TinyFish performed',
|
||||
},
|
||||
errors: {
|
||||
type: 'array',
|
||||
description: 'Fields that did not match the requested schema',
|
||||
items: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
path: { type: 'string', description: 'Path to the failing field' },
|
||||
expected: { type: 'string', description: 'Expected type or constraint' },
|
||||
received: { type: 'string', description: 'Type actually returned' },
|
||||
message: { type: 'string', description: 'Validation error message' },
|
||||
},
|
||||
},
|
||||
},
|
||||
} as const
|
||||
|
||||
/** Output property descriptions for a run summary, shared by get-run and list-runs. */
|
||||
export const RUN_SUMMARY_OUTPUT_PROPERTIES = {
|
||||
runId: { type: 'string', description: 'Run identifier' },
|
||||
status: {
|
||||
type: 'string',
|
||||
description: 'PENDING, RUNNING, COMPLETED, FAILED, or CANCELLED',
|
||||
},
|
||||
goal: { type: 'string', description: 'Natural-language goal the run was given' },
|
||||
createdAt: { type: 'string', description: 'ISO 8601 timestamp when the run was created' },
|
||||
startedAt: {
|
||||
type: 'string',
|
||||
description: 'ISO 8601 timestamp when the run started executing',
|
||||
optional: true,
|
||||
},
|
||||
finishedAt: {
|
||||
type: 'string',
|
||||
description: 'ISO 8601 timestamp when the run finished',
|
||||
optional: true,
|
||||
},
|
||||
numOfSteps: {
|
||||
type: 'number',
|
||||
description: 'Steps taken, null while the run is still in progress',
|
||||
optional: true,
|
||||
},
|
||||
result: {
|
||||
type: 'json',
|
||||
description: 'Structured data the agent extracted, null until the run succeeds',
|
||||
optional: true,
|
||||
},
|
||||
schemaValidation: {
|
||||
type: 'object',
|
||||
description: 'Validation of the result against the requested output schema',
|
||||
optional: true,
|
||||
properties: SCHEMA_VALIDATION_OUTPUT_PROPERTIES,
|
||||
},
|
||||
error: {
|
||||
type: 'object',
|
||||
description: 'Failure details, null while the run is pending or succeeded',
|
||||
optional: true,
|
||||
properties: RUN_ERROR_OUTPUT_PROPERTIES,
|
||||
},
|
||||
streamingUrl: {
|
||||
type: 'string',
|
||||
description: 'Live browser view URL, available while the run is executing',
|
||||
optional: true,
|
||||
},
|
||||
browserConfig: {
|
||||
type: 'object',
|
||||
description: 'Proxy settings the run executed with',
|
||||
optional: true,
|
||||
properties: {
|
||||
proxyEnabled: { type: 'boolean', description: 'Whether a proxy was used', optional: true },
|
||||
proxyCountryCode: { type: 'string', description: 'Proxy country code', optional: true },
|
||||
},
|
||||
},
|
||||
} as const
|
||||
@@ -0,0 +1,297 @@
|
||||
import { getErrorMessage } from '@sim/utils/errors'
|
||||
import type {
|
||||
TinyFishBrowserConfig,
|
||||
TinyFishRawBrowserConfig,
|
||||
TinyFishRawRunError,
|
||||
TinyFishRawRunSummary,
|
||||
TinyFishRawSchemaValidation,
|
||||
TinyFishRunError,
|
||||
TinyFishRunSummary,
|
||||
TinyFishSchemaValidation,
|
||||
} from '@/tools/tinyfish/types'
|
||||
|
||||
/** Agent API host. Search, Fetch, and Browser each live on their own host. */
|
||||
export const TINYFISH_AGENT_API_BASE = 'https://agent.tinyfish.ai'
|
||||
|
||||
/** Search API host. */
|
||||
export const TINYFISH_SEARCH_API_BASE = 'https://api.search.tinyfish.ai'
|
||||
|
||||
/** Fetch API host. */
|
||||
export const TINYFISH_FETCH_API_BASE = 'https://api.fetch.tinyfish.ai'
|
||||
|
||||
/**
|
||||
* Identifies Sim to TinyFish's analytics on every automation request, as the
|
||||
* `api_integration` field documents for integration partners.
|
||||
*/
|
||||
export const TINYFISH_API_INTEGRATION = 'sim'
|
||||
|
||||
/** Maximum URLs the Fetch API accepts in one request. */
|
||||
export const MAX_FETCH_URLS = 10
|
||||
|
||||
/** Every TinyFish surface authenticates with the same `X-API-Key` header. */
|
||||
export function tinyfishHeaders(apiKey: string): Record<string, string> {
|
||||
return {
|
||||
'Content-Type': 'application/json',
|
||||
'X-API-Key': apiKey,
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Extracts the message from TinyFish's API-level error envelope.
|
||||
*
|
||||
* Both documented shapes nest the message under `error`, so an unparseable or
|
||||
* differently shaped body falls back to the HTTP status line.
|
||||
*/
|
||||
export async function tinyfishErrorMessage(response: Response): Promise<string> {
|
||||
try {
|
||||
const body = await response.json()
|
||||
const message = body?.error?.message
|
||||
if (typeof message === 'string' && message.length > 0) {
|
||||
const code = body?.error?.code
|
||||
return typeof code === 'string' && code.length > 0 ? `${code}: ${message}` : message
|
||||
}
|
||||
} catch {}
|
||||
return `TinyFish request failed with status ${response.status}`
|
||||
}
|
||||
|
||||
/** Splits a comma- or newline-separated list into trimmed, non-empty entries. */
|
||||
export function parseList(input: string | string[] | undefined): string[] {
|
||||
if (!input) return []
|
||||
const values = Array.isArray(input) ? input : input.split(/[\n,]/)
|
||||
return values.map((value) => String(value).trim()).filter(Boolean)
|
||||
}
|
||||
|
||||
/**
|
||||
* Accepts a JSON Schema as either a parsed object or the stringified form the
|
||||
* block's code editor produces, and returns it as an object.
|
||||
*/
|
||||
export function parseJsonSchema(
|
||||
input: string | Record<string, unknown> | undefined
|
||||
): Record<string, unknown> | undefined {
|
||||
if (!input) return undefined
|
||||
|
||||
if (typeof input !== 'string') {
|
||||
if (Array.isArray(input)) {
|
||||
throw new Error('Output Schema must be a JSON object')
|
||||
}
|
||||
return input
|
||||
}
|
||||
|
||||
const trimmed = input.trim()
|
||||
if (!trimmed) return undefined
|
||||
|
||||
let parsed: unknown
|
||||
try {
|
||||
parsed = JSON.parse(trimmed)
|
||||
} catch (error) {
|
||||
throw new Error(`Output Schema is not valid JSON: ${getErrorMessage(error)}`)
|
||||
}
|
||||
|
||||
if (typeof parsed !== 'object' || parsed === null || Array.isArray(parsed)) {
|
||||
throw new Error('Output Schema must be a JSON object')
|
||||
}
|
||||
return parsed as Record<string, unknown>
|
||||
}
|
||||
|
||||
export function mapSchemaValidation(
|
||||
raw: TinyFishRawSchemaValidation | null | undefined
|
||||
): TinyFishSchemaValidation | null {
|
||||
if (!raw) return null
|
||||
return {
|
||||
valid: raw.valid ?? false,
|
||||
rePromptAttempts: raw.re_prompt_attempts ?? 0,
|
||||
errors: (raw.errors ?? []).map((issue) => ({
|
||||
path: issue?.path ?? '',
|
||||
expected: issue?.expected ?? '',
|
||||
received: issue?.received ?? '',
|
||||
message: issue?.message ?? '',
|
||||
})),
|
||||
}
|
||||
}
|
||||
|
||||
export function mapRunError(raw: TinyFishRawRunError | null | undefined): TinyFishRunError | null {
|
||||
if (!raw) return null
|
||||
return {
|
||||
code: raw.code ?? null,
|
||||
message: raw.message ?? '',
|
||||
category: raw.category ?? 'UNKNOWN',
|
||||
retryAfter: raw.retry_after ?? null,
|
||||
helpUrl: raw.help_url ?? null,
|
||||
helpMessage: raw.help_message ?? null,
|
||||
}
|
||||
}
|
||||
|
||||
function mapBrowserConfig(
|
||||
raw: TinyFishRawBrowserConfig | null | undefined
|
||||
): TinyFishBrowserConfig | null {
|
||||
if (!raw) return null
|
||||
return {
|
||||
proxyEnabled: raw.proxy_enabled ?? null,
|
||||
proxyCountryCode: raw.proxy_country_code ?? null,
|
||||
}
|
||||
}
|
||||
|
||||
/** Maps a run object from `GET /v1/runs` or `GET /v1/runs/{id}` to Sim's camelCase shape. */
|
||||
export function mapRunSummary(raw: TinyFishRawRunSummary): TinyFishRunSummary {
|
||||
return {
|
||||
runId: raw.run_id ?? '',
|
||||
status: raw.status ?? 'PENDING',
|
||||
goal: raw.goal ?? '',
|
||||
createdAt: raw.created_at ?? '',
|
||||
startedAt: raw.started_at ?? null,
|
||||
finishedAt: raw.finished_at ?? null,
|
||||
numOfSteps: raw.num_of_steps ?? null,
|
||||
result: raw.result ?? null,
|
||||
schemaValidation: mapSchemaValidation(raw.schema_validation),
|
||||
error: mapRunError(raw.error),
|
||||
streamingUrl: raw.streaming_url ?? null,
|
||||
browserConfig: mapBrowserConfig(raw.browser_config),
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Builds the request body shared by `POST /v1/automation/run` and
|
||||
* `POST /v1/automation/run-async`.
|
||||
*
|
||||
* Optional fields are omitted rather than sent as null so TinyFish applies its
|
||||
* own documented defaults (`lite` browser profile, `default` agent mode, 150
|
||||
* max steps, no proxy, no vault).
|
||||
*/
|
||||
export function buildAutomationBody(params: {
|
||||
url: string
|
||||
goal: string
|
||||
browserProfile?: string
|
||||
agentMode?: string
|
||||
maxSteps?: number
|
||||
outputSchema?: string | Record<string, unknown>
|
||||
proxyEnabled?: boolean
|
||||
proxyCountryCode?: string
|
||||
useVault?: boolean
|
||||
credentialItemIds?: string | string[]
|
||||
webhookUrl?: string
|
||||
}): Record<string, unknown> {
|
||||
const body: Record<string, unknown> = {
|
||||
url: params.url,
|
||||
goal: params.goal,
|
||||
api_integration: TINYFISH_API_INTEGRATION,
|
||||
}
|
||||
|
||||
if (params.browserProfile) body.browser_profile = params.browserProfile
|
||||
|
||||
const agentConfig: Record<string, unknown> = {}
|
||||
if (params.agentMode) agentConfig.mode = params.agentMode
|
||||
if (typeof params.maxSteps === 'number' && Number.isFinite(params.maxSteps)) {
|
||||
agentConfig.max_steps = params.maxSteps
|
||||
}
|
||||
if (Object.keys(agentConfig).length > 0) body.agent_config = agentConfig
|
||||
|
||||
if (params.proxyEnabled) {
|
||||
const proxyConfig: Record<string, unknown> = { enabled: true, type: 'tetra' }
|
||||
if (params.proxyCountryCode) proxyConfig.country_code = params.proxyCountryCode
|
||||
body.proxy_config = proxyConfig
|
||||
}
|
||||
|
||||
const outputSchema = parseJsonSchema(params.outputSchema)
|
||||
if (outputSchema) body.output_schema = outputSchema
|
||||
|
||||
if (params.useVault) {
|
||||
body.use_vault = true
|
||||
const credentialItemIds = parseList(params.credentialItemIds)
|
||||
if (credentialItemIds.length > 0) body.credential_item_ids = credentialItemIds
|
||||
}
|
||||
|
||||
if (params.webhookUrl) body.webhook_url = params.webhookUrl
|
||||
|
||||
return body
|
||||
}
|
||||
|
||||
/** Param declarations shared by the synchronous and asynchronous automation tools. */
|
||||
export const AUTOMATION_TOOL_PARAMS = {
|
||||
url: {
|
||||
type: 'string',
|
||||
required: true,
|
||||
visibility: 'user-or-llm',
|
||||
description: 'Target website URL the agent starts on',
|
||||
},
|
||||
goal: {
|
||||
type: 'string',
|
||||
required: true,
|
||||
visibility: 'user-or-llm',
|
||||
description: 'Natural-language description of what to accomplish on the website',
|
||||
},
|
||||
browserProfile: {
|
||||
type: 'string',
|
||||
required: false,
|
||||
visibility: 'user-or-llm',
|
||||
description: 'Browser engine: "lite" (standard) or "stealth" (anti-detection)',
|
||||
},
|
||||
agentMode: {
|
||||
type: 'string',
|
||||
required: false,
|
||||
visibility: 'user-or-llm',
|
||||
description: 'Agent behavior: "default" or "strict" (fail fast)',
|
||||
},
|
||||
maxSteps: {
|
||||
type: 'number',
|
||||
required: false,
|
||||
visibility: 'user-or-llm',
|
||||
description: 'Maximum tool-call steps before the agent stops (1-500, default 150)',
|
||||
},
|
||||
outputSchema: {
|
||||
type: 'json',
|
||||
required: false,
|
||||
visibility: 'user-or-llm',
|
||||
description: 'JSON Schema draft-07 contract the run result must satisfy',
|
||||
},
|
||||
proxyEnabled: {
|
||||
type: 'boolean',
|
||||
required: false,
|
||||
visibility: 'user-or-llm',
|
||||
description: 'Route the run through TinyFish’s Tetra proxy',
|
||||
},
|
||||
proxyCountryCode: {
|
||||
type: 'string',
|
||||
required: false,
|
||||
visibility: 'user-or-llm',
|
||||
description: 'Proxy country: US, GB, CA, DE, FR, JP, or AU',
|
||||
},
|
||||
useVault: {
|
||||
type: 'boolean',
|
||||
required: false,
|
||||
visibility: 'user-only',
|
||||
description: 'Let the run use credentials from the connected TinyFish vault',
|
||||
},
|
||||
credentialItemIds: {
|
||||
type: 'string',
|
||||
required: false,
|
||||
visibility: 'user-only',
|
||||
description: 'Comma-separated vault credential URIs to scope the run to',
|
||||
},
|
||||
apiKey: {
|
||||
type: 'string',
|
||||
required: true,
|
||||
visibility: 'user-only',
|
||||
description: 'TinyFish API key',
|
||||
},
|
||||
} as const
|
||||
|
||||
/**
|
||||
* Model-visible selection shared by the automation tools.
|
||||
*
|
||||
* TinyFish hands the goal to the agent's LLM verbatim, and re-prompts that same
|
||||
* model with the output schema when a result does not satisfy it — the run's
|
||||
* `schema_validation.re_prompt_attempts` counts exactly those repair passes. Both
|
||||
* are therefore model input and are projected to canonical secret placeholders
|
||||
* before the request is formatted.
|
||||
*
|
||||
* The target URL, browser profile, proxy settings, and vault scoping are ordinary
|
||||
* request inputs and keep their normal semantics. The schema is projected as the
|
||||
* whole param rather than through `applyProjected` because it reaches the request
|
||||
* formatter unparsed, matching `tools/exa/search.ts`.
|
||||
*/
|
||||
export function selectAutomationModelInput(params: {
|
||||
goal?: string
|
||||
outputSchema?: string | Record<string, unknown>
|
||||
}) {
|
||||
return { goal: params.goal, outputSchema: params.outputSchema }
|
||||
}
|
||||
@@ -21,6 +21,7 @@ export type BYOKProviderId =
|
||||
| 'firecrawl'
|
||||
| 'exa'
|
||||
| 'context_dev'
|
||||
| 'tinyfish'
|
||||
| 'serper'
|
||||
| 'jina'
|
||||
| 'perplexity'
|
||||
|
||||
@@ -54,6 +54,7 @@
|
||||
"check:utils": "bun run scripts/check-utils-enforcement.ts",
|
||||
"check:canvas-sentences": "bun run apps/sim/scripts/check-canvas-sentences.ts --require-coverage",
|
||||
"check:bare-icons": "bun run scripts/check-bare-icons.ts",
|
||||
"check:byok-providers": "bun run scripts/check-byok-providers.ts",
|
||||
"check:icon-paths": "bun run scripts/check-icon-paths.ts",
|
||||
"check:icon-path-precision": "bun run scripts/check-icon-path-precision.ts",
|
||||
"check:migrations": "bun run scripts/check-migrations-safety.ts",
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
{
|
||||
"updatedAt": "2026-08-26",
|
||||
"updatedAt": "2026-08-27",
|
||||
"integrations": [
|
||||
{
|
||||
"type": "onepassword",
|
||||
@@ -23498,6 +23498,57 @@
|
||||
"integrationType": "analytics",
|
||||
"tags": ["data-warehouse", "data-analytics"]
|
||||
},
|
||||
{
|
||||
"type": "tinyfish",
|
||||
"slug": "tinyfish",
|
||||
"name": "TinyFish",
|
||||
"description": "Automate and read the live web",
|
||||
"longDescription": "Integrate TinyFish into the workflow. Give a web agent a natural-language goal and let it drive a real browser on any site, queue and track long-running automations, search the web, and fetch pages as clean markdown.",
|
||||
"bgColor": "#FF6700",
|
||||
"iconName": "TinyFishIcon",
|
||||
"docsUrl": "https://docs.sim.ai/integrations/tinyfish",
|
||||
"operations": [
|
||||
{
|
||||
"name": "Run Agent",
|
||||
"description": "Run a TinyFish web agent against a website and wait for it to finish, returning the structured result it extracted"
|
||||
},
|
||||
{
|
||||
"name": "Start Agent Run",
|
||||
"description": "Queue a TinyFish web agent run and return its run ID immediately, without waiting for the automation to finish"
|
||||
},
|
||||
{
|
||||
"name": "Get Run",
|
||||
"description": "Get the status, extracted result, and step history of a TinyFish automation run by its ID"
|
||||
},
|
||||
{
|
||||
"name": "Cancel Run",
|
||||
"description": "Cancel a queued or in-progress TinyFish automation run by its ID"
|
||||
},
|
||||
{
|
||||
"name": "List Runs",
|
||||
"description": "List TinyFish automation runs, optionally filtered by status, goal text, or creation date"
|
||||
},
|
||||
{
|
||||
"name": "Search",
|
||||
"description": "Search the web with TinyFish and get ranked results with titles, snippets, and URLs"
|
||||
},
|
||||
{
|
||||
"name": "Fetch URLs",
|
||||
"description": "Fetch up to 10 URLs with TinyFish, rendering JavaScript when needed, and return clean extracted content"
|
||||
},
|
||||
{
|
||||
"name": "List Vault Items",
|
||||
"description": "List the credentials available from password managers connected to TinyFish, with the URIs an agent run can be scoped to"
|
||||
}
|
||||
],
|
||||
"operationCount": 8,
|
||||
"triggers": [],
|
||||
"triggerCount": 0,
|
||||
"authType": "api-key",
|
||||
"category": "tools",
|
||||
"integrationType": "ai",
|
||||
"tags": ["web-scraping", "automation", "agentic"]
|
||||
},
|
||||
{
|
||||
"type": "trello",
|
||||
"slug": "trello",
|
||||
|
||||
@@ -0,0 +1,206 @@
|
||||
#!/usr/bin/env bun
|
||||
/**
|
||||
* Audits that every BYOK provider is wired through all four places it must appear.
|
||||
*
|
||||
* A hosted tool names its provider once, in `hosting.byokProviderId`, but that id
|
||||
* has to be registered in three other files before the feature actually works:
|
||||
*
|
||||
* tools/types.ts the `BYOKProviderId` union tools compile against
|
||||
* lib/api/contracts/byok-keys.ts the zod enum the byok-keys route validates against
|
||||
* settings/.../byok.tsx `PROVIDERS` the row the settings page renders
|
||||
* settings/.../byok.tsx `SECTIONS` the section that row is grouped under
|
||||
*
|
||||
* Only the first is enforced by the compiler. The other three fail *silently*:
|
||||
*
|
||||
* - Missing from `PROVIDERS`, the settings page has no row, so a workspace can
|
||||
* never bring its own key and is stuck on the hosted key.
|
||||
* - Missing from `PROVIDER_SECTIONS`, the row exists but the sectioned renderer
|
||||
* (`byok-key-manager.tsx` filters `providers` by `section.ids.includes(p.id)`)
|
||||
* drops it, so the page looks correct in source and renders nothing.
|
||||
* - Drifted between the two `BYOKProviderId` declarations, a tool can name a
|
||||
* provider the route then rejects at runtime.
|
||||
*
|
||||
* None of those produce a type error, a test failure, or a log line — which is
|
||||
* exactly why they need an audit rather than a convention.
|
||||
*
|
||||
* Run: `bun run check:byok-providers`
|
||||
*/
|
||||
import { readFile } from 'node:fs/promises'
|
||||
import { dirname, resolve } from 'node:path'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import { tools } from '../apps/sim/tools/registry'
|
||||
|
||||
const SCRIPT_DIR = dirname(fileURLToPath(import.meta.url))
|
||||
const ROOT = resolve(SCRIPT_DIR, '..')
|
||||
const APP = resolve(ROOT, 'apps/sim')
|
||||
const TOOL_TYPES = resolve(APP, 'tools/types.ts')
|
||||
const CONTRACT = resolve(APP, 'lib/api/contracts/byok-keys.ts')
|
||||
const SETTINGS = resolve(APP, 'app/workspace/[workspaceId]/settings/components/byok/byok.tsx')
|
||||
|
||||
/** Path as written in an error message, relative to the repo root. */
|
||||
function rel(absolute: string): string {
|
||||
return absolute.slice(ROOT.length + 1)
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns the source between the brackets opened by the first match of `start`.
|
||||
*
|
||||
* Bracket-counting rather than a lazy regex: every one of these blocks nests
|
||||
* (an object per provider, an array per section), so `[\s\S]*?\]` would stop at
|
||||
* the first inner close.
|
||||
*/
|
||||
function blockAfter(source: string, start: RegExp, open: '[' | '{'): string {
|
||||
const match = source.match(start)
|
||||
if (match?.index === undefined) {
|
||||
throw new Error(`could not locate ${start} — has the declaration been renamed?`)
|
||||
}
|
||||
const close = open === '[' ? ']' : '}'
|
||||
const from = source.indexOf(open, match.index + match[0].length - 1)
|
||||
if (from === -1) throw new Error(`no ${open} after ${start}`)
|
||||
|
||||
let depth = 0
|
||||
for (let i = from; i < source.length; i++) {
|
||||
if (source[i] === open) depth++
|
||||
else if (source[i] === close) {
|
||||
depth--
|
||||
if (depth === 0) return source.slice(from + 1, i)
|
||||
}
|
||||
}
|
||||
throw new Error(`unbalanced ${open} after ${start}`)
|
||||
}
|
||||
|
||||
/** Every single-quoted string literal in a chunk of source, in order. */
|
||||
function quoted(source: string): string[] {
|
||||
return [...source.matchAll(/'([a-z0-9_-]+)'/gi)].map((m) => m[1])
|
||||
}
|
||||
|
||||
interface Failure {
|
||||
file: string
|
||||
message: string
|
||||
items: string[]
|
||||
fix: string
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const [toolTypesSrc, contractSrc, settingsSrc] = await Promise.all([
|
||||
readFile(TOOL_TYPES, 'utf8'),
|
||||
readFile(CONTRACT, 'utf8'),
|
||||
readFile(SETTINGS, 'utf8'),
|
||||
])
|
||||
|
||||
const unionDecl = toolTypesSrc.match(/export type BYOKProviderId =([\s\S]*?)\n\n/)
|
||||
if (!unionDecl) throw new Error(`could not locate BYOKProviderId union in ${rel(TOOL_TYPES)}`)
|
||||
const union = new Set(quoted(unionDecl[1]))
|
||||
|
||||
const schema = new Set(quoted(blockAfter(contractSrc, /byokProviderIdSchema = z\.enum\(/, '[')))
|
||||
|
||||
const settingsProviders = new Set(
|
||||
[
|
||||
...blockAfter(settingsSrc, /const PROVIDERS[^=]*=/, '[').matchAll(
|
||||
/\bid:\s*'([a-z0-9_-]+)'/gi
|
||||
),
|
||||
].map((m) => m[1])
|
||||
)
|
||||
|
||||
const sectioned = new Set(
|
||||
[
|
||||
...blockAfter(settingsSrc, /const PROVIDER_SECTIONS[^=]*=/, '[').matchAll(
|
||||
/\bids:\s*\[([\s\S]*?)\]/g
|
||||
),
|
||||
].flatMap((m) => quoted(m[1]))
|
||||
)
|
||||
|
||||
/** Provider id -> the hosted tools that name it. */
|
||||
const hostedBy = new Map<string, string[]>()
|
||||
for (const [toolId, tool] of Object.entries(tools)) {
|
||||
const provider = tool.hosting?.byokProviderId
|
||||
if (!provider) continue
|
||||
const existing = hostedBy.get(provider)
|
||||
if (existing) existing.push(toolId)
|
||||
else hostedBy.set(provider, [toolId])
|
||||
}
|
||||
|
||||
const failures: Failure[] = []
|
||||
const describe = (provider: string) => {
|
||||
const owners = hostedBy.get(provider) ?? []
|
||||
return owners.length > 0
|
||||
? `${provider} (${owners[0]}${owners.length > 1 ? ', …' : ''})`
|
||||
: provider
|
||||
}
|
||||
|
||||
const missingFromSchema = [...hostedBy.keys()].filter((p) => !schema.has(p)).sort()
|
||||
if (missingFromSchema.length > 0) {
|
||||
failures.push({
|
||||
file: rel(CONTRACT),
|
||||
message: 'hosted tools name a provider the byok-keys route would reject',
|
||||
items: missingFromSchema.map(describe),
|
||||
fix: 'add the id to byokProviderIdSchema',
|
||||
})
|
||||
}
|
||||
|
||||
const missingFromSettings = [...hostedBy.keys()]
|
||||
.filter((p) => schema.has(p) && !settingsProviders.has(p))
|
||||
.sort()
|
||||
if (missingFromSettings.length > 0) {
|
||||
failures.push({
|
||||
file: rel(SETTINGS),
|
||||
message:
|
||||
'hosted tools name a provider with no settings row, so a workspace cannot bring its own key',
|
||||
items: missingFromSettings.map(describe),
|
||||
fix: 'add an entry to PROVIDERS',
|
||||
})
|
||||
}
|
||||
|
||||
const unsectioned = [...settingsProviders].filter((p) => !sectioned.has(p)).sort()
|
||||
if (unsectioned.length > 0) {
|
||||
failures.push({
|
||||
file: rel(SETTINGS),
|
||||
message: 'PROVIDERS entries the sectioned renderer drops, so their row never appears',
|
||||
items: unsectioned,
|
||||
fix: 'add the id to the right PROVIDER_SECTIONS section',
|
||||
})
|
||||
}
|
||||
|
||||
const orphanedSections = [...sectioned].filter((p) => !settingsProviders.has(p)).sort()
|
||||
if (orphanedSections.length > 0) {
|
||||
failures.push({
|
||||
file: rel(SETTINGS),
|
||||
message: 'PROVIDER_SECTIONS lists ids with no matching PROVIDERS entry',
|
||||
items: orphanedSections,
|
||||
fix: 'remove the stale id, or add the missing PROVIDERS entry',
|
||||
})
|
||||
}
|
||||
|
||||
const unionOnly = [...union].filter((p) => !schema.has(p)).sort()
|
||||
const schemaOnly = [...schema].filter((p) => !union.has(p)).sort()
|
||||
if (unionOnly.length > 0 || schemaOnly.length > 0) {
|
||||
failures.push({
|
||||
file: `${rel(TOOL_TYPES)} vs ${rel(CONTRACT)}`,
|
||||
message: 'the two BYOKProviderId declarations have drifted',
|
||||
items: [
|
||||
...unionOnly.map((p) => `${p} (union only)`),
|
||||
...schemaOnly.map((p) => `${p} (zod enum only)`),
|
||||
],
|
||||
fix: 'keep the union and the zod enum listing the same ids',
|
||||
})
|
||||
}
|
||||
|
||||
if (failures.length > 0) {
|
||||
console.error('\n❌ BYOK provider wiring is incomplete\n')
|
||||
for (const failure of failures) {
|
||||
console.error(` ${failure.file}: ${failure.message}`)
|
||||
for (const item of failure.items) console.error(` - ${item}`)
|
||||
console.error(` fix: ${failure.fix}\n`)
|
||||
}
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
console.log(
|
||||
`✓ BYOK provider wiring is complete (${hostedBy.size} hosted providers, ${settingsProviders.size} settings rows)`
|
||||
)
|
||||
}
|
||||
|
||||
main().catch((error) => {
|
||||
console.error(`\n❌ check-byok-providers failed: ${error.message}`)
|
||||
process.exit(1)
|
||||
})
|
||||
Reference in New Issue
Block a user