Kyle Carberry
32217259b7
## Problem
Local tool results were persisted and replayed to the model verbatim,
with no size cap. A single oversized result, most often a multi-megabyte
response from an MCP tool, overflows the prompt on the next request.
Every retry rebuilds the same history and fails the same way, leaving
the chat wedged in `error`. Auto-compaction is reactive (token usage is
only known after a response), so it can't catch a single result that
blows the very next request.
## Fix
Cap every locally-executed tool result at its single choke point,
`executeSingleTool` in `chatloop`, so the cap covers built-in tools,
**global (deployment-pinned) MCP**, **workspace MCP**, and provider
runners uniformly. Because this runs before the result is published to
the live stream and before it is committed, the SSE preview, the
persisted message, and the model replay all see the same bounded output.
The budget is token-aware: a single tool result may use at most half the
model's context window (`~4 bytes/token`), with a `16KB` floor and a
`64KB` default when the window is unknown. Truncation keeps the head and
tail of the output and replaces the middle with a marker telling the
model how much was removed and to narrow its query; it is UTF-8 safe and
never exceeds the budget. Binary media `Data` is passed through
untouched (only the text payload is bounded).
A `coderd_chatd_tool_result_truncated_total{provider,model,tool_name}`
counter and a warning log record each truncation.
## Out of scope
- Provider-executed results (e.g. web search) arrive via the stream, not
`executeSingleTool`.
- Dynamic/external tool results submitted through the `/tool-results`
API are validated as JSON elsewhere.
- Cumulative growth across many results is still handled by context
compaction; this change only bounds any single result.
<details>
<summary>Implementation notes</summary>
- New `coderd/x/chatd/chatloop/tooltruncate.go`:
`toolResultByteBudget(contextLimitTokens)` and
`truncateToolResultText(text, maxBytes)` (pure, unit-tested).
- `chatloop.go`: added `ContextLimit` to `ExecuteLocalToolsOptions`;
threaded a computed byte budget through `executeTools` into
`executeSingleTool`, where `resp.Content` is capped for the text,
media-text, and error branches.
- `generation.go`: passes `ContextLimit: prepared.ContextLimitFallback`
(the model's configured context limit).
- `metrics.go`: new `ToolResultTruncatedTotal` counter +
`RecordToolResultTruncated`.
- Tunable knobs live as constants in `tooltruncate.go`
(`toolResultContextDivisor = 2`, `bytesPerTokenEstimate`,
`minToolResultBytes`, `defaultToolResultBytes`).
Verified: `go build ./coderd/x/chatd/...`, `go test
./coderd/x/chatd/chatloop/...`, and the `chatd` test binary compiles.
</details>
---
Resolves CODAGT-678
Generated by Coder Agents on behalf of @kylecarbs.
Metrics Documentation Generator
This tool generates the Prometheus metrics documentation at docs/admin/integrations/prometheus.md.
How It Works
The documentation is generated from two metrics files:
metrics(static, manually maintained)generated_metrics(auto-generated, do not edit)
These files are merged and used to produce the final documentation.
metrics (static)
Contains metrics that are not directly defined in the coder source code:
go_*: Go runtime metricsprocess_*: Process metrics from prometheus/client_golangpromhttp_*: Prometheus HTTP handler metricscoder_ai_gateway_*: AI Gateway metrics are registered through prefixed registerer that the scanner does not resolve.
Note
This file also contains edge cases where metric metadata cannot be accurately extracted by the scanner (e.g., labels determined by runtime logic). Static metrics take priority over generated metrics when both files contain the same metric name.
Edit this file to add metrics that should appear in the documentation but are not scanned from the coder codebase,
or to manually override metrics where the scanner generates incorrect metadata (e.g., missing runtime-determined labels like in agent_scripts_executed_total).
generated_metrics (auto-generated)
Contains metrics extracted from the coder source code by the AST scanner (scanner/scanner.go).
Do not edit this file directly. It is regenerated by running:
make scripts/metricsdocgen/generated_metrics
Updating Metrics Documentation
To regenerate the documentation after code changes:
make docs/admin/integrations/prometheus.md
This will:
- Run the scanner to update
generated_metrics - Merge
metricsandgenerated_metricsmetric files - Update the documentation file