opensource/coder
3
0
Fork 0
mirror of https://github.com/coder/coder.git synced 2026-09-24 15:04:27 +08:00
Code Issues Packages Projects Releases Wiki Activity

feat: defer MCP tool schemas behind a find_tools search (#28225)

Browse Source 
## Summary

When the `mcp-tool-search` experiment is enabled, chatd stops inlining
connected MCP tool schemas into every generation. It instead exposes a
built-in `find_tools` tool whose description carries a compact catalog
of the deferred tools, and only ships full JSON schemas for tools the
model has activated by searching or by calling them directly.

Closes [CODAGT-760](https://linear.app/coder/issue/CODAGT-760).

## Problem

Tool-heavy agent configurations (GitHub, Linear, Notion, and dev-tooling
MCP servers) inline over 100k tokens of tool schema definitions into
every generation. Initial uncached requests reached ~216k tokens with
time-to-first-token close to nine minutes, while the model typically
invokes only a handful of tools per turn.

## How it works

- `decideMCPToolSearch` defers external and workspace `.mcp.json` MCP
tools whenever the experiment is enabled. Native, dynamic, provider,
skill, and transport tools are never deferred.
- `find_tools` embeds a server-grouped catalog in its tool description
(degrading to names-only, then counts-only, then a constant-size summary
past a context-scaled size cap) and scores keyword matches across tool
names, descriptions, parameter schemas, and server metadata. Queries can
scope to one server with a `server:` prefix, and exact `names` arguments
always activate.
- Activation state is ephemeral: it is re-derived each generation from
surviving chat history (`find_tools` results and direct calls to
deferred tools), so activations naturally lapse when compaction
summarizes them away. Aggregate activated schema weight is capped at 10%
of the context window, shedding the least recently activated schemas
first; `find_tools` shares that budget across parallel calls in one
step. No new persistence.
- Deferred tools stay registered for execution, so the model can call a
cataloged tool directly without searching first; the schema is activated
for subsequent steps.
- Fail-open: the experiment being disabled, an empty candidate set, or
an MCP tool named `find_tools` all disable deferral, leaving today's
behavior byte-identical on the wire.
- Prometheus counters/histograms track `find_tools` calls, matches,
activations, and deferred token weight.
- The conversation timeline renders `find_tools` calls with a collapsed
search summary and expandable match list, falling back to the generic
renderer on malformed payloads.

## Validation

- Unit tests for the catalog, matcher, experiment-gated decision, and
activation derivation; end-to-end chatd generation tests covering
search-then-call, direct-call activation, experiment-off wire parity,
compaction lapse, and subagent tool gating.
- Storybook interaction tests for the timeline rendering and
malformed-payload fallback.
- Remote dogfood UAT on dev.coder.com passed: deferral with a real MCP
server and Anthropic model, direct calls without prior search,
activation persistence across turns, experiment-off parity, and clean
UI/console.

> Disclosure: Mux (AI agent) authored this PR on Mike's behalf.
This commit is contained in:
Michael Suchacz
2026-08-18 19:12:47 +02:00
committed by GitHub
parent 62f4afbb60
commit 7724ee281a
26 changed files with 3762 additions and 59 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/admin/integrations/prometheus.md
+4
View File
@@ -240,6 +240,10 @@ The `coder_ai_gateway_cost_control_*` metrics are exported only by `coderd`.
| `coderd_chatd_agents_queued_for_capacity` | gauge | Deployment-wide number of chats waiting for a concurrent-agent capacity slot. Every replica reports the same database-derived value; aggregate with max, not sum. | `pool` |
| `coderd_chatd_chats` | gauge | Number of chats being processed, by state. | `state` |
| `coderd_chatd_compaction_total` | counter | Total compaction outcomes (only recorded when compaction was triggered or failed). | `model` `provider` `result` |
| `coderd_chatd_find_tools_activations_total` | counter | Total deferred tool activations returned by find_tools. | |
| `coderd_chatd_find_tools_calls_total` | counter | Total find_tools calls. | |
| `coderd_chatd_find_tools_empty_total` | counter | Total find_tools calls with no matches. | |
| `coderd_chatd_find_tools_match_count` | histogram | Number of matches returned by find_tools calls. | |
| `coderd_chatd_hook_context_size_bytes` | histogram | Lifecycle hook model context response size in bytes. | `event` |
| `coderd_chatd_hook_decisions_total` | counter | Total lifecycle hook permission decisions by event and decision. | `decision` `event` |
| `coderd_chatd_hook_dispatch_seconds` | histogram | Lifecycle hook dispatch duration in seconds. | `event` |