mirror of
https://github.com/coder/coder.git
synced 2026-09-22 05:05:20 +08:00
Thomas Kosiewski
aa89801ee5
Closes CODAGT-592.
## Problem
The advisor tool sometimes fails with the opaque error `advisor produced
no text output`. Live reproduction against `claude-sonnet-4-6` showed
the cause: `BuildAdvisorMessages` forwards the parent conversation's raw
`tool_use`/`tool_result` blocks into the nested advisor call, which
defines no tools. The nested model imitates the forwarded pattern and
spends its turn committing to a tool call it cannot make (captured
reasoning from a failing run: "The user wants me to make another tool
call to the advisor about writing a poem about cucumbers."), so the step
ends with reasoning-only or empty content and no advice. Because each
chat step currently rebuilds the advisor runtime and snapshot
(CODAGT-593), the second advisor call in a run reliably sees the first
call's exchange, which is why the first call succeeds and later ones
fail.
## Fix
- `BuildAdvisorMessages` rewrites tool activity as plain-text notes:
assistant tool-call parts are removed and folded, together with their
matching result, into a single user-role note of the form `[The parent
agent ran the X tool with input {...}. Result: ...]`. No raw tool blocks
and no bare call lines reach the tool-less nested request. This also
removes the provider requirement that `tool_result` blocks pair with a
`tool_use`, so results orphaned by window truncation are kept as notes
instead of dropped.
- The `advisor produced no text output` error now appends the finish
reason and content-part kinds, e.g. `advisor produced no text output
(finish_reason=stop; parts: reasoning=1)`, so field reports distinguish
tool-call mimicry, reasoning-only turns, and truncation.
Validated live by driving the production `RunAdvisor` path against
`claude-sonnet-4-6` through the dev.coder.com AI gateway: the failing
scenario went from 3/3 errors to 6/6 genuine advice (with and without
extended thinking), with the control scenario unaffected.
Related: CODAGT-593 (per-step advisor runtime recreation, addressed
separately) and CODAGT-742 (advisor tool call design).
<details>
<summary>Investigation and validation details</summary>
### Reproduction
A CLI prototype constructed the exact conversation snapshot the
generation preparer hands the advisor tool and called the real
`chatadvisor.NewRuntime` / `Runtime.RunAdvisor` / `BuildAdvisorMessages`
/ `chatloop.GenerateAssistant` chain against live `claude-sonnet-4-6`,
with a stream-teeing model wrapper capturing what `runner.go` discards
(finish reason, part kinds, reasoning text).
| Scenario (snapshot contents) | Thinking | Before fix | After fix |
|---|---|---|---|
| control: call #1 state, no prior advisor exchange | on | 3/3 advice |
2/2 advice |
| repro: call #2 state, prior advisor `tool_use`/`tool_result` pair
forwarded | on | 3/3 `advisor produced no text output` | 3/3 genuine
advice |
| repro | off | 2/3 same error, 1/3 degenerate advice ("I'll ask the
advisor...") | 3/3 genuine advice |
Every failing response was a tiny thinking block, zero text, zero
tool-call stream parts, finish reason `stop`; the model's own reasoning
text showed it deciding to "make the second tool call" in a request with
`tools=0`. The refunded `remaining_uses: 1200` in the failing
tool-result JSON matches the original issue screenshot.
### Decision log
- Tool exchanges are folded into a single user-role note per call/result
pair. A first attempt rendered assistant-authored `[tool call:
name(input)]` text lines plus separate result messages; live runs then
returned the literal `[tool call: advisor(...)]` line as the advice 6/6
times. The bare assistant call line is itself an imitable pattern, so no
assistant-authored tool artifact may survive the handoff. The folded
user-role note produced 6/6 genuine advice.
- An assistant message that carried only tool calls is dropped entirely;
the folded notes preserve the information.
- `dropOrphanToolMessages` was removed: without raw tool blocks there is
no provider pairing constraint, and an orphaned result note retains
context value.
- A reasoning-budget-starvation hypothesis (thinking budget consuming
`MaxOutputTokens`) did not reproduce on `claude-sonnet-4-6`; the model
adapts thinking length to the cap. The enriched error would identify
such cases on other models via `finish_reason=length`.
- CODAGT-593 (persisting the advisor runtime across steps) is
intentionally not addressed here; it shrinks the priming window but the
handoff fix is what removes the failure mode.
</details>
---
*This PR was generated by Coder Agents on behalf of @ThomasK33 (Linear
agent session for CODAGT-592).*