From b3607f51b257ec4a91e6e1ea7dbebece9e31eea0 Mon Sep 17 00:00:00 2001 From: Michael Suchacz <203725896+ibetitsmike@users.noreply.github.com> Date: Wed, 12 Aug 2026 17:08:01 +0200 Subject: [PATCH] fix: keep chat token usage for streams ending with usage-less chunks (#28068) Bumps the coder/fantasy fork pin to pick up coder/fantasy#52. ## Problem Chats on `poolside/laguna-xs-2.1` showed no context usage: every assistant message persisted NULL token columns, and automatic compaction never triggered, so chats ran to context overflow. AIBridge recorded correct usage for the same requests, so the loss was client-side in fantasy. When tools are declared, laguna-xs reports cumulative usage on every delta chunk and ends the stream with a `finish_reason` chunk whose `usage` is null, with no trailing usage-only chunk. Fantasy's chat-completions stream loops reassigned usage from every chunk, and the default stream usage hook returns zero usage for usage-less chunks, so the trailing finish chunk wiped the real usage one chunk earlier. The Finish part then reported `Usage{0,0,0}`, which chatd persists as NULL (`nullInt64IfNonZero`). ## Fix coder/fantasy#52 adopts the stream usage hook's result only when the chunk actually carries usage, in both the chat-completions stream loop and the JSON-mode object stream loop. This mirrors the aibridge fix in #27967, which is why the gateway recorded usage correctly while fantasy lost it. Spec-compliant backends that emit usage once on the final chunk are unaffected. This PR pins the fork at the merged commit and documents the fork-only patch in the go.mod comment block. ## Validation - coder/fantasy#52: new regression tests for both stream loops, proven red against the unguarded code; full module tests, vet, gofmt, and golangci-lint green; fork CI green before merge. - Here: `go build ./...`, `go vet ./coderd/x/chatd/...`, and `go test ./coderd/x/chatd/...` (including the chatdebug field-coverage guard) all pass with the bumped pin. > Mux acted on Mike's behalf to create this PR. --- go.mod | 9 ++++++--- go.sum | 4 ++-- 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/go.mod b/go.mod index 6312e77dd7..fe2d42b84b 100644 --- a/go.mod +++ b/go.mod @@ -96,10 +96,13 @@ replace github.com/spf13/afero => github.com/aslilac/afero v0.0.0-20250403163713 // region the request is actually signed for. // 6) coder/fantasy#50, report per-request Anthropic usage at Finish so // multi-iteration requests do not inflate context estimates. -// 7) go.mod replaces pointing anthropic-sdk-go and openai-go at the +// 7) coder/fantasy#52, keep the last usage-bearing chunk when an +// OpenAI-compatible stream ends with a usage-less finish chunk, so +// trailing chunks cannot zero reported usage. +// 8) go.mod replaces pointing anthropic-sdk-go and openai-go at the // coder forks below. -// See: https://github.com/coder/fantasy/commits/6f8df3735907 -replace charm.land/fantasy => github.com/coder/fantasy v0.0.0-20260812075759-6f8df3735907 +// See: https://github.com/coder/fantasy/commits/833854abc543 +replace charm.land/fantasy => github.com/coder/fantasy v0.0.0-20260812122040-833854abc543 // coder/coder uses a fork of charmbracelet's fork of the Anthropic Go SDK // with performance improvements and Bedrock header cleanup. diff --git a/go.sum b/go.sum index 8755033c4b..29f5022cc5 100644 --- a/go.sum +++ b/go.sum @@ -327,8 +327,8 @@ github.com/coder/bubbletea v1.2.2-0.20241212190825-007a1cdb2c41 h1:SBN/DA63+ZHwu github.com/coder/bubbletea v1.2.2-0.20241212190825-007a1cdb2c41/go.mod h1:I9ULxr64UaOSUv7hcb3nX4kowodJCVS7vt7VVJk/kW4= github.com/coder/clistat v1.2.1 h1:P9/10njXMyj5cWzIU5wkRsSy5LVQH49+tcGMsAgWX0w= github.com/coder/clistat v1.2.1/go.mod h1:m7SC0uj88eEERgvF8Kn6+w6XF21BeSr+15f7GoLAw0A= -github.com/coder/fantasy v0.0.0-20260812075759-6f8df3735907 h1:Mirl1lw+0MsndEL0feesyIqtxTdpiKrR8AH4zcA6SZU= -github.com/coder/fantasy v0.0.0-20260812075759-6f8df3735907/go.mod h1:lcL8B/uroFq5YDHNFij2an0yarOtD7C3rmzD31i8oTA= +github.com/coder/fantasy v0.0.0-20260812122040-833854abc543 h1:GDVwVOgilrtKfUZGN0L2hANHJbQEL0R1gBNYMJ0TnMA= +github.com/coder/fantasy v0.0.0-20260812122040-833854abc543/go.mod h1:lcL8B/uroFq5YDHNFij2an0yarOtD7C3rmzD31i8oTA= github.com/coder/flog v1.1.0 h1:kbAes1ai8fIS5OeV+QAnKBQE22ty1jRF/mcAwHpLBa4= github.com/coder/flog v1.1.0/go.mod h1:UQlQvrkJBvnRGo69Le8E24Tcl5SJleAAR7gYEHzAmdQ= github.com/coder/go-httpstat v0.0.0-20230801153223-321c88088322 h1:m0lPZjlQ7vdVpRBPKfYIFlmgevoTkBxB10wv6l2gOaU=