## Summary
When the `mcp-tool-search` experiment is enabled, chatd stops inlining
connected MCP tool schemas into every generation. It instead exposes a
built-in `find_tools` tool whose description carries a compact catalog
of the deferred tools, and only ships full JSON schemas for tools the
model has activated by searching or by calling them directly.
Closes [CODAGT-760](https://linear.app/coder/issue/CODAGT-760).
## Problem
Tool-heavy agent configurations (GitHub, Linear, Notion, and dev-tooling
MCP servers) inline over 100k tokens of tool schema definitions into
every generation. Initial uncached requests reached ~216k tokens with
time-to-first-token close to nine minutes, while the model typically
invokes only a handful of tools per turn.
## How it works
- `decideMCPToolSearch` defers external and workspace `.mcp.json` MCP
tools whenever the experiment is enabled. Native, dynamic, provider,
skill, and transport tools are never deferred.
- `find_tools` embeds a server-grouped catalog in its tool description
(degrading to names-only, then counts-only, then a constant-size summary
past a context-scaled size cap) and scores keyword matches across tool
names, descriptions, parameter schemas, and server metadata. Queries can
scope to one server with a `server:` prefix, and exact `names` arguments
always activate.
- Activation state is ephemeral: it is re-derived each generation from
surviving chat history (`find_tools` results and direct calls to
deferred tools), so activations naturally lapse when compaction
summarizes them away. Aggregate activated schema weight is capped at 10%
of the context window, shedding the least recently activated schemas
first; `find_tools` shares that budget across parallel calls in one
step. No new persistence.
- Deferred tools stay registered for execution, so the model can call a
cataloged tool directly without searching first; the schema is activated
for subsequent steps.
- Fail-open: the experiment being disabled, an empty candidate set, or
an MCP tool named `find_tools` all disable deferral, leaving today's
behavior byte-identical on the wire.
- Prometheus counters/histograms track `find_tools` calls, matches,
activations, and deferred token weight.
- The conversation timeline renders `find_tools` calls with a collapsed
search summary and expandable match list, falling back to the generic
renderer on malformed payloads.
## Validation
- Unit tests for the catalog, matcher, experiment-gated decision, and
activation derivation; end-to-end chatd generation tests covering
search-then-call, direct-call activation, experiment-off wire parity,
compaction lapse, and subagent tool gating.
- Storybook interaction tests for the timeline rendering and
malformed-payload fallback.
- Remote dogfood UAT on dev.coder.com passed: deferral with a real MCP
server and Anthropic model, direct calls without prior search,
activation persistence across turns, experiment-off parity, and clean
UI/console.
> Disclosure: Mux (AI agent) authored this PR on Mike's behalf.
Removes OpenAI Responses "chain mode" from chatd. Closes CODAGT-445.
- Deletes `chatopenai/responses.go` (chain detection, activation, prompt filtering, response ID extraction) and its tests.
- Deletes the `ChainBroken` classification in `chaterror` and the chatloop retry bookkeeping that disabled chain mode mid-generation.
- Drops the `chain_broken` label from the `coderd_chatd_stream_retries_total` metric.
- Stops reading and writing `chat_messages.provider_response_id`
- Deletes the dead `ClearChatMessageProviderResponseIDsByChatID` query. Dropping the column is a follow-up migration.
- Deletes three chatloop hooks no caller sets (`ReloadMessages`, `DisableChainMode`, `PrepareMessages`), the dead `const AgentChatContextSentinelPath`, and stale chain-mode comments.
🤖 Generated by Coder Agents on behalf of @johnstcn.
## Problem
Local tool results were persisted and replayed to the model verbatim,
with no size cap. A single oversized result, most often a multi-megabyte
response from an MCP tool, overflows the prompt on the next request.
Every retry rebuilds the same history and fails the same way, leaving
the chat wedged in `error`. Auto-compaction is reactive (token usage is
only known after a response), so it can't catch a single result that
blows the very next request.
## Fix
Cap every locally-executed tool result at its single choke point,
`executeSingleTool` in `chatloop`, so the cap covers built-in tools,
**global (deployment-pinned) MCP**, **workspace MCP**, and provider
runners uniformly. Because this runs before the result is published to
the live stream and before it is committed, the SSE preview, the
persisted message, and the model replay all see the same bounded output.
The budget is token-aware: a single tool result may use at most half the
model's context window (`~4 bytes/token`), with a `16KB` floor and a
`64KB` default when the window is unknown. Truncation keeps the head and
tail of the output and replaces the middle with a marker telling the
model how much was removed and to narrow its query; it is UTF-8 safe and
never exceeds the budget. Binary media `Data` is passed through
untouched (only the text payload is bounded).
A `coderd_chatd_tool_result_truncated_total{provider,model,tool_name}`
counter and a warning log record each truncation.
## Out of scope
- Provider-executed results (e.g. web search) arrive via the stream, not
`executeSingleTool`.
- Dynamic/external tool results submitted through the `/tool-results`
API are validated as JSON elsewhere.
- Cumulative growth across many results is still handled by context
compaction; this change only bounds any single result.
<details>
<summary>Implementation notes</summary>
- New `coderd/x/chatd/chatloop/tooltruncate.go`:
`toolResultByteBudget(contextLimitTokens)` and
`truncateToolResultText(text, maxBytes)` (pure, unit-tested).
- `chatloop.go`: added `ContextLimit` to `ExecuteLocalToolsOptions`;
threaded a computed byte budget through `executeTools` into
`executeSingleTool`, where `resp.Content` is capped for the text,
media-text, and error branches.
- `generation.go`: passes `ContextLimit: prepared.ContextLimitFallback`
(the model's configured context limit).
- `metrics.go`: new `ToolResultTruncatedTotal` counter +
`RecordToolResultTruncated`.
- Tunable knobs live as constants in `tooltruncate.go`
(`toolResultContextDivisor = 2`, `bytesPerTokenEstimate`,
`minToolResultBytes`, `defaultToolResultBytes`).
Verified: `go build ./coderd/x/chatd/...`, `go test
./coderd/x/chatd/chatloop/...`, and the `chatd` test binary compiles.
</details>
---
Resolves CODAGT-678
Generated by Coder Agents on behalf of @kylecarbs.
When OpenAI's Responses API returns `Previous response with id ... not
found` for a chained turn, classify it as a `ChainBroken` retry, clear
`previous_response_id`, exit chain mode, reload full history, and let
`chatretry` retry. Self-heals chats whose anchor was poisoned before
#25074 stopped truncated streams from being persisted as a successful
turn with a stored response id.
The new state is exposed via the existing
`coderd_chatd_stream_retries_total` counter as a
`chain_broken="true"|"false"` label. Aggregating queries (`sum`, `rate`
over `provider`/`model`/`kind`) keep working without changes; raw-series
matchers without aggregation will now see two series per `(provider,
model, kind)` where they previously saw one. The metric is internal-only
so the blast radius should be small, but if you have dashboards that
index by exact label matchers without aggregation they will need an
extra `sum` or an explicit `chain_broken` selector.
> 🤖 This PR was created with the help of Coder Agents, and was reviewed by a human 🧑💻
Moves the chat error kind taxonomy from `coderd/x/chatd/chaterror` into
`codersdk.ChatErrorKind` and types `ChatError.Kind` /
`ChatStreamRetry.Kind` so generated TypeScript exposes an SDK-owned
union, including `usage_limit`. Backend chat classification now
references the SDK constants directly while preserving the existing JSON
string values.
Keeps chat usage-limit admission failures on their existing 409 response
shape. The frontend maps structured usage-limit responses to the
SDK-owned `usage_limit` kind, uses generated `TypesGen.ChatErrorKind`
directly, and removes the local string union and alias.
Adds production-observability metrics to coderd/x/chatd/ for
model-level correlation and a chatStreams memory-leak investigation.
- Label per-request chatd metrics (steps_total, message_count,
prompt_size_bytes, tool_result_size_bytes, ttft_seconds,
compaction_total) with `model` and enrich the per-turn logger
with provider/model.
- Add `coderd_chatd_stream_retries_total{provider, model, kind}`
counter incremented in chatloop before OnRetry.
- Register a prometheus.Collector exposing `streams_active`,
`stream_buffer_size_max`, `stream_buffer_events`,
`stream_subscribers` from p.chatStreams.
- Add `coderd_chatd_stream_buffer_dropped_total` counter,
incremented per publishToStream drop independently of the
existing log-rate-limited bufferDropCount.
- Snapshot logger/model before the title-generation goroutine to
avoid a data race with the logger/model rebind below it.
> 🤖
The `coderd_chatd_message_count` histogram's current max bucket of 128
is being hit in production. This increases the exponential bucket count
from 8 to 11, extending coverage from `1..128` to `1..1024`.
Before: `1, 2, 4, 8, 16, 32, 64, 128`
After: `1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024`
Co-authored-by: blink-so[bot] <211532188+blink-so[bot]@users.noreply.github.com>
Michael Suchacz