Commit Graph
283 Commits
Author SHA1 Message Date
Erik Engervall 6445ff393d Update .gitignore to include .pnpm-store and refine OAuth protected resource URL handling in index.ts. Improved error handling and added comments for clarity in the getOAuthProtectedResourceMetadataUrl function. 2026-05-27 10:36:54 +02:00
Erik Engervall 2795094866 Merge remote-tracking branch 'origin/main' into noaa/oauth_ 2026-05-27 09:20:03 +02:00
Eric Ciarla 9b6a8b68a9 Merge pull request #240 from hmishra2250/fix/mcp-search-instructions
Add MCP instructions for Firecrawl search routing
2026-05-26 10:35:54 -04:00
Erik Engervall 8853f78141 Refactor OAuth token introspection handling in index.ts. Updated type definitions and modified API key handling to improve clarity and consistency. Changed header authorization method for token introspection requests. 2026-05-26 16:20:29 +02:00
Erik Engervall 0234f66837 Update firecrawl-fastmcp dependency to version 1.0.5 and enhance OAuth credential handling in the FastMCP server. Introduced new functions for OAuth issuer and resource URL management, and improved error handling for token introspection. Updated type definitions to support new OAuth options. 2026-05-25 22:24:19 +02:00
Erik Engervall be98368e65 Merge remote-tracking branch 'origin/main' into noaa/oauth_ 2026-05-25 22:05:31 +02:00
Himadri MandClaude Opus 4.6 386df8e05a fix: remove scanner-flagged line from MCP instructions
Remove "Do not use built-in web search unless firecrawl_search is
unavailable, returns an error, or returns no results for the query"
from the instructions field. This line was flagged by Jonathan/Mogery
as triggering MCP scanner vendor-steering detection.

Validated with 30 tests (15 Claude Code, 15 Codex): routing rates
are identical with and without the line. The positive framing alone
is sufficient.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-25 22:51:57 +05:30
Himadri MandClaude Opus 4.6 7dedc4ab72 Add MCP instructions for Firecrawl search routing
Users who set up Firecrawl via MCP (not firecrawl init) get 0% search
routing because the model picks built-in search before seeing the
firecrawl_search tool. This adds an instructions field to the MCP
server initialize response so clients surface it to the model at
decision time, before tool selection.

Validated with real Firecrawl API across 3 rounds of 5 queries each:
- Claude Code: 15/15 no built-in WebSearch (100%)
- Codex: Firecrawl used in 10/15 (67%)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-25 14:17:08 +05:30
Michael OliverandNicolas 869cab1def fix(search): remove regex lookahead from domain schema pattern (#225)
* fix(search): remove regex lookahead from domain schema pattern

The searchDomainSchema used a regex with a lookahead assertion
(?=.{1,253}$) to validate total hostname length. JSON Schema's
pattern keyword does not support lookahead assertions (per ECMA 262
subset), causing LLM providers to reject the tool schema with:

  Regex lookahead (?=...) is not supported in JSON Schema pattern

Replace the lookahead with Zod's .min(1).max(253) constraints, which
translate to minLength/maxLength in JSON Schema. The regex itself
still validates hostname label format correctly.

Relates to #112, #196

* Update package.json

---------

Co-authored-by: Nicolas <20311743+nickscamara@users.noreply.github.com>
2026-05-20 11:01:47 -07:00
Developers DigestandNicolas 1227968cf3 Add Firecrawl monitor tools (#234)
* Add Firecrawl monitor tools

Introduce a new monitor tools module (src/monitor.ts) and register it from src/index.ts. The module implements direct /v2/monitor HTTP calls (monitorRequest) — including auth resolution and query handling — to work around the SDK's injected `origin` field that breaks monitor endpoints. It registers several MCP tools for monitors (create, list, get, update, delete, run, checks, check) that serialize responses as JSON text. Also adds the import and registerMonitorTools(server) call before server.start.

* Update monitor.ts

* Update package.json

* Nick:

---------

Co-authored-by: Nicolas <20311743+nickscamara@users.noreply.github.com>
2026-05-17 12:53:12 -04:00
Eric Ciarla c8473d0088 Update README.md 2026-05-15 12:38:12 -04:00
Erik Engervall 67898b42db OAuth support 2026-05-15 11:29:26 -04:00
Nicolas 0925b76b08 (feat/search) Search feedback (#236)
* Nick: search feedback

* Update index.ts

* Update package.json
2026-05-14 15:31:10 -04:00
mogeryandCursor 71aa555b6e feat(scrape): add query mode option
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-05 17:54:12 +02:00
Nicolas 57d2d0f31c Delete registry.yml 2026-05-01 12:21:02 -03:00
Nicolas 17022873da Update package.json 2026-05-01 12:19:34 -03:00
Nicolas 2bab322e32 Update publish.yml 2026-05-01 12:14:01 -03:00
Gergő MóriczandNicolas 3b57f4ff7e feat(search): includeDomains/excldueDomains (#222)
* feat(search): includeDomains/excldueDomains

* Nick:

* Update publish.yml

---------

Co-authored-by: Nicolas <20311743+nickscamara@users.noreply.github.com>
2026-05-01 12:12:15 -03:00
firecrawl-spring[bot]andmicahstairs badde9be68 feat(scrape): add audio format to scrape tool (#223)
Co-authored-by: firecrawl-spring[bot] <254786068+firecrawl-spring[bot]@users.noreply.github.com>
Co-authored-by: micahstairs <micah@sideguide.dev>
2026-04-30 19:40:08 -04:00
Erik Engervall bcbb8d576d Merge pull request #216 from firecrawl/noaa/add-annotations-to-tools
Add annotations to firecrawl tools for improved usability
2026-04-28 15:12:34 +02:00
Erik Engervall c74f943ebf Enhance firecrawl_parse tool with annotations for improved usability. Added title, read-only, and open world hints to facilitate user understanding of the tool's functionality. 2026-04-28 15:11:10 +02:00
Erik Engervall d625e3b522 Merge remote-tracking branch 'origin/main' into noaa/add-annotations-to-tools 2026-04-28 15:09:57 +02:00
Gergő Móricz 974c01e258 Merge pull request #215 from firecrawl/mog/parse-api
feat: add parse (ENG-4830)
2026-04-27 17:24:45 +02:00
Abimael Martell 400072efb6 Merge pull request #218 from firecrawl/abimaelmartell/lockdown-scrape
add lockdown param to firecrawl_scrape
2026-04-22 15:30:14 -07:00
Abimael Martell dfa92ccfc2 add lockdown param to firecrawl_scrape 2026-04-22 15:26:58 -07:00
Erik Engervall 3525dccef4 Add annotations to firecrawl tools for improved usability
This commit introduces annotations for various firecrawl tools in the index.ts file, providing titles and hints regarding their usage, such as read-only and open world capabilities. Additionally, the fastmcp.d.ts file is updated to include a new annotations property in the tool interface, detailing the expected structure for these annotations.
2026-04-21 23:26:17 +02:00
mogery 3788f1ec57 feat: add parse fn 2026-04-21 18:05:57 +01:00
Eric Ciarla a90089f050 Minor copy updates 2026-04-14 20:27:27 -04:00
Nicolas 6fef044379 (feat/interact) Interact (#193)
* Nick:

* Update index.ts

* Update index.ts

* Update README.md

* Nick:
2026-03-23 17:05:47 -03:00
Developers Digest d267d9beb2 expand firecrawl_agent not-recommended list to reduce false triggers 2026-03-13 12:56:43 -04:00
Developers Digest b97e107358 add query format support (#187) (#190)
* add query format support to scrape tool

* bump version to 3.12.0

* prefer markdown over query format in scrape tool description

MCP agents can read and reason over scraped content directly —
the query format is redundant when the agent already has the
scrape result. default guidance now recommends markdown.
2026-03-12 15:59:01 -07:00
Nicolas 655c9f6635 Update package.json 2026-03-12 10:02:42 -07:00
Nicolas b4a4347c45 Update package.json 2026-03-12 10:01:12 -07:00
Developers Digest 4e03036c98 Merge pull request #188 from firecrawl/revert-185-remove-agent
Revert "Bump version; remove firecrawl agent tools"
2026-03-12 16:54:46 +00:00
Developers Digest cd168a2b3b Revert "Bump version; remove firecrawl agent tools" 2026-03-12 12:54:32 -04:00
Developers Digest 9585ec36ce Merge pull request #185 from firecrawl/remove-agent
Bump version; remove firecrawl agent tools
2026-03-12 00:22:25 +00:00
Developers Digest 8ea8263f5e Bump version; remove firecrawl agent tools
Update package version to 3.11.0 and remove the asynchronous agent features from the server API. The firecrawl_agent and firecrawl_agent_status tool registrations (and their detailed usage/polling docs) were deleted from src/index.ts and the usage guidance was simplified to prefer firecrawl_map + scrape (use map when scrape fails). Consumers should stop calling the removed agent endpoints and migrate workflows to map+scrape or other supported tools.
2026-03-11 20:11:30 -04:00
Nicolas caf6e6b235 Nick: bump version 2026-03-03 12:22:47 -08:00
Nicolas d86de741d0 Revert "Nick: integration"
This reverts commit 68278765d9.
2026-03-03 10:43:38 -08:00
Nicolas 68278765d9 Nick: integration 2026-03-03 10:41:12 -08:00
Leonardo Grigorio a44a3c2386 Merge pull request #165 from firecrawl/fix/gemini-anyof-schema-compat
fix(schema): replace anyOf unions with flat params for Gemini compatibility
2026-03-03 09:42:46 -03:00
Developers Digest 1435e45469 Add browser profile support and bump version (#171) 2026-02-26 14:39:43 -08:00
firecrawl-spring[bot] 4e8da309a3 fix(schema): replace anyOf unions with flat params for Gemini compatibility
Google Gemini's function calling API rejects tool schemas that use
anyOf alongside other fields. This replaces all z.union() usages with
flat parameter alternatives:

- formats: now a simple string enum array (added 'json' to the enum)
- jsonOptions: new param for JSON extraction prompt/schema
- screenshotOptions: new param for screenshot fullPage/quality/viewport
- parsers: now a simple string enum array
- pdfOptions: new param for PDF maxPages
- webhook (crawl): now a simple string, with separate webhookHeaders

Helper functions (buildFormatsArray, buildParsersArray, buildWebhook,
transformScrapeParams) reassemble the nested format the API expects
from the flat MCP params at execution time.

Closes #164
2026-02-20 04:09:52 +00:00
Nicolas d2af56feb8 Update pnpm-lock.yaml 2026-02-16 18:57:06 -08:00
Nicolas 7dd9674a0d Update package.json 2026-02-16 18:55:54 -08:00
Developers DigestandNicolas 998d27b6fa b (#162)
* init browser

* Bump firecrawl and update session API

Upgrade @mendable/firecrawl-js to 4.13.0-beta.3 and update the session API: rename TTL parameters (ttlTotal -> ttl, ttlWithoutActivity -> activityTtl), adjust documentation and zod schemas/types accordingly, and change the execute-language options by replacing 'js' with 'node' and adding 'bash' (including examples and types).

* Add npm tag logic; update browser execute docs

CI: Change publish workflow to run on pushes to main when package.json changes and keep manual dispatch. Add a step to determine npm tag (beta if version contains "beta", otherwise latest) and pass that tag to pnpm publish. Only run MCP publisher install/login/publish steps when the tag is "latest" to avoid publishing prereleases to the MCP registry.

Code: Update firecrawl_browser_execute docs and parameter ordering in src/index.ts. The description now recommends using bash agent-browser commands by default (with examples and command reference), provides a Python example for Playwright scripting, and adjusts the language enum/default ordering accordingly. These docs clarify usage and preferred workflows for browser automation.

* revert beta release ci back

* update readme

* Update package.json

---------

Co-authored-by: Nicolas <20311743+nickscamara@users.noreply.github.com>
2026-02-16 18:54:26 -08:00
Leonardo Grigorio 0159dda1b0 Merge pull request #157 from firecrawl/feat/scrape-map-fallback-guidance
docs(scrape): add map tool as fallback step for SPA/JS-heavy pages
2026-02-05 13:27:48 -08:00
firecrawl-spring[bot]andleonardogrig ab3292ec1c docs(map): emphasize map+scrape workflow before using agent
- Enhanced map tool description to highlight its role in finding
  specific page URLs when scrape returns empty results
- Added search parameter example showing the recommended workflow
- Removed waitFor from default scrape JSON example (it's optional,
  only needed when JS rendering requires extra wait time)

Co-Authored-By: leonardogrig <leo@sideguide.dev>
2026-02-05 20:59:51 +00:00
firecrawl-spring[bot]andleonardogrig 051b49d5f5 docs(scrape): add map tool as fallback step for SPA/JS-heavy pages
When scrape returns empty or minimal content from JavaScript-rendered
pages, guide users to try firecrawl_map with a search parameter to
discover the specific page URL before falling back to firecrawl_agent.

This is more efficient than immediately using the agent, as many large
documentation sites spread content across multiple URLs that can be
discovered via map.

Co-Authored-By: leonardogrig <leo@sideguide.dev>
2026-02-05 20:55:56 +00:00
Leonardo Grigorio 72b0f4ab7c Merge pull request #156 from firecrawl/fix/json-format-preference-and-spa-recovery
fix(tools): improve JSON format preference and SPA handling
2026-02-05 12:39:14 -08:00