fix: keep chat token usage for streams ending with usage-less chunks (#28068)

Bumps the coder/fantasy fork pin to pick up coder/fantasy#52.

## Problem

Chats on `poolside/laguna-xs-2.1` showed no context usage: every
assistant message persisted NULL token columns, and automatic compaction
never triggered, so chats ran to context overflow. AIBridge recorded
correct usage for the same requests, so the loss was client-side in
fantasy.

When tools are declared, laguna-xs reports cumulative usage on every
delta chunk and ends the stream with a `finish_reason` chunk whose
`usage` is null, with no trailing usage-only chunk. Fantasy's
chat-completions stream loops reassigned usage from every chunk, and the
default stream usage hook returns zero usage for usage-less chunks, so
the trailing finish chunk wiped the real usage one chunk earlier. The
Finish part then reported `Usage{0,0,0}`, which chatd persists as NULL
(`nullInt64IfNonZero`).

## Fix

coder/fantasy#52 adopts the stream usage hook's result only when the
chunk actually carries usage, in both the chat-completions stream loop
and the JSON-mode object stream loop. This mirrors the aibridge fix in
#27967, which is why the gateway recorded usage correctly while fantasy
lost it. Spec-compliant backends that emit usage once on the final chunk
are unaffected.

This PR pins the fork at the merged commit and documents the fork-only
patch in the go.mod comment block.

## Validation

- coder/fantasy#52: new regression tests for both stream loops, proven
red against the unguarded code; full module tests, vet, gofmt, and
golangci-lint green; fork CI green before merge.
- Here: `go build ./...`, `go vet ./coderd/x/chatd/...`, and `go test
./coderd/x/chatd/...` (including the chatdebug field-coverage guard) all
pass with the bumped pin.

> Mux acted on Mike's behalf to create this PR.
This commit is contained in:
Michael Suchacz
2026-08-12 17:08:01 +02:00
committed by GitHub
parent 5a33b669b4
commit b3607f51b2
2 changed files with 8 additions and 5 deletions
+6 -3
View File
@@ -96,10 +96,13 @@ replace github.com/spf13/afero => github.com/aslilac/afero v0.0.0-20250403163713
// region the request is actually signed for.
// 6) coder/fantasy#50, report per-request Anthropic usage at Finish so
// multi-iteration requests do not inflate context estimates.
// 7) go.mod replaces pointing anthropic-sdk-go and openai-go at the
// 7) coder/fantasy#52, keep the last usage-bearing chunk when an
// OpenAI-compatible stream ends with a usage-less finish chunk, so
// trailing chunks cannot zero reported usage.
// 8) go.mod replaces pointing anthropic-sdk-go and openai-go at the
// coder forks below.
// See: https://github.com/coder/fantasy/commits/6f8df3735907
replace charm.land/fantasy => github.com/coder/fantasy v0.0.0-20260812075759-6f8df3735907
// See: https://github.com/coder/fantasy/commits/833854abc543
replace charm.land/fantasy => github.com/coder/fantasy v0.0.0-20260812122040-833854abc543
// coder/coder uses a fork of charmbracelet's fork of the Anthropic Go SDK
// with performance improvements and Bedrock header cleanup.
+2 -2
View File
@@ -327,8 +327,8 @@ github.com/coder/bubbletea v1.2.2-0.20241212190825-007a1cdb2c41 h1:SBN/DA63+ZHwu
github.com/coder/bubbletea v1.2.2-0.20241212190825-007a1cdb2c41/go.mod h1:I9ULxr64UaOSUv7hcb3nX4kowodJCVS7vt7VVJk/kW4=
github.com/coder/clistat v1.2.1 h1:P9/10njXMyj5cWzIU5wkRsSy5LVQH49+tcGMsAgWX0w=
github.com/coder/clistat v1.2.1/go.mod h1:m7SC0uj88eEERgvF8Kn6+w6XF21BeSr+15f7GoLAw0A=
github.com/coder/fantasy v0.0.0-20260812075759-6f8df3735907 h1:Mirl1lw+0MsndEL0feesyIqtxTdpiKrR8AH4zcA6SZU=
github.com/coder/fantasy v0.0.0-20260812075759-6f8df3735907/go.mod h1:lcL8B/uroFq5YDHNFij2an0yarOtD7C3rmzD31i8oTA=
github.com/coder/fantasy v0.0.0-20260812122040-833854abc543 h1:GDVwVOgilrtKfUZGN0L2hANHJbQEL0R1gBNYMJ0TnMA=
github.com/coder/fantasy v0.0.0-20260812122040-833854abc543/go.mod h1:lcL8B/uroFq5YDHNFij2an0yarOtD7C3rmzD31i8oTA=
github.com/coder/flog v1.1.0 h1:kbAes1ai8fIS5OeV+QAnKBQE22ty1jRF/mcAwHpLBa4=
github.com/coder/flog v1.1.0/go.mod h1:UQlQvrkJBvnRGo69Le8E24Tcl5SJleAAR7gYEHzAmdQ=
github.com/coder/go-httpstat v0.0.0-20230801153223-321c88088322 h1:m0lPZjlQ7vdVpRBPKfYIFlmgevoTkBxB10wv6l2gOaU=