fix: tighten single tool result byte budget (#26763)

## Problem

#26637 caps each locally-executed tool result (built-in,
global/deployment MCP, workspace MCP) at a per-result byte budget
derived from the model's context window. The budget was `ContextLimit/2
* 4 bytes` — i.e. **half the window at an optimistic 4 bytes/token**.

On large-context models that is far too generous. With a
`1,000,000`-token `ContextLimit` the per-result cap is **~2 MB**. A user
hit exactly this with a chatd (deployment-pinned) MCP tool: the result
was truncated to **1,998,709 characters** and still overflowed the
prompt. 2 MB of dense text (JSON/logs/code) is ~650k–1M tokens — most or
all of the window for a *single* result — so the cap fired but didn't
actually prevent the overflow.

## Fix

Tighten the two budget constants in `tooltruncate.go`:

| constant | before | after |
| --- | --- | --- |
| `toolResultContextDivisor` | `2` (½ window) | `3` (⅓ window) |
| `bytesPerTokenEstimate` | `4` | `3` (conservative) |

The budget becomes `ContextLimit/3 * 3 ≈ ContextLimit` bytes:

| ContextLimit | before | after |
| --- | --- | --- |
| 1,000,000 | ~2 MB | ~1 MB |
| 200,000 | ~400 KB | ~200 KB |
| unknown (≤0) | 64 KB | 64 KB (unchanged) |

The 16 KB floor and 64 KB unknown-window default are unchanged. A
conservative bytes-per-token estimate is intentional: dense payloads run
well under 4 B/tok, so a lower estimate yields a smaller byte budget
that is less likely to underestimate the true token cost.

No behavioral code paths change — only the two constants and their doc
comments. The existing `tooltruncate_internal_test.go` cases derive
their expectations from the constants (`LargeWindow`) or exercise the
floor/default (`BelowFloor`, `Unknown`), so they remain green.

<details>
<summary>Investigation notes</summary>

Global/deployment MCP tools (`mcpclient.ConnectAll`) are appended to
`prepared.Tools` and execute locally via `ExecuteLocalTools →
executeTools → executeSingleTool`, so the #26637 cap *does* apply to
them for text results (`convertCallResult` joins text content into
`resp.Content`). The cap was simply too large:
`toolResultByteBudget(ContextLimit)` = `ContextLimit/2*4` ≈ 2 MB for a
1M-token window. Reverse-engineering the reported `1,998,709` truncated
characters confirms a `ContextLimit` of ~1,000,000 tokens.

Known gaps left for follow-ups (out of scope here):
- **Per-step aggregate is unbounded.** MCP tools advertise `Parallel:
true` and `executeSingleTool` caps each result independently, so N
parallel calls in one step can sum to N × the per-result cap.
- **Binary/media `Data` bypasses the cap.** Only the text payload is
bounded; `image`/`media`/blob embedded-resource results are
base64-encoded untouched in `executeSingleTool`.
- **Compaction is reactive.** It is gated on the prior step's reported
usage (`latestPromptUsage`), so it can't pre-empt a single large result
appended on the current step.
</details>

---

Generated by Coder Agents on behalf of @kylecarbs.
This commit is contained in:
Kyle Carberry
2026-06-26 13:33:55 -06:00
committed by GitHub
parent 8ff2109298
commit d6daf273aa
+12 -6
View File
@@ -7,19 +7,25 @@ import (
const (
// toolResultContextDivisor bounds how much of a model's context
// window a single tool result may occupy: at most 1/N of the
// window a single tool result may occupy: at most 1/3 of the
// window. This caps a single oversized result (most often a large
// MCP response) so it cannot overflow the prompt on its own, while
// still letting a generous amount of output through. Cumulative
// still letting a useful amount of output through. The divisor is
// deliberately larger than 2 so the absolute cap stays sane on
// large-context (e.g. 1M-token) models, where a more generous
// fraction would still admit multi-megabyte results. Cumulative
// growth across many results is handled separately by context
// compaction.
toolResultContextDivisor = 2
toolResultContextDivisor = 3
// bytesPerTokenEstimate converts a token budget into a byte budget.
// Tool output is capped before tokenization, so this is a coarse,
// provider-agnostic estimate. Roughly 4 bytes per token for typical
// text.
bytesPerTokenEstimate = 4
// provider-agnostic estimate. It is deliberately conservative at ~3
// bytes per token: dense payloads (JSON, logs, code, non-ASCII) run
// well below 4 bytes per token, so the lower estimate yields a
// smaller byte budget that is less likely to underestimate the true
// token cost.
bytesPerTokenEstimate = 3
// minToolResultBytes is the floor for the per-result byte budget so
// small or unknown context windows still let useful output through.