GatewayMediaExecutor.download() left the fetch Response body un-read and un-cancelled on several early-exit paths: a terminal non-ok HTTP status, an oversized declared content-length (declaredLength > maxApiArtifactBytes), and a mid-stream writeSync failure in the download loop. In each case the underlying undici connection/socket is leaked, so repeated failed downloads accumulate open connections and exhaust the pool.
Cancel the response body before throwing on those paths, and cancel the reader in the write-error handler, matching the cancellation already done on the retry branch and the in-loop size-limit branch.
Requests converted to the OpenAI Responses protocol left the outbound
body without any session-stable field: prompt_cache_key was never set
and the inbound Anthropic metadata.user_id was dropped. Multi-channel
Responses upstreams that pin sessions on body fields hashed each turn
onto a different channel, so channel-bound encrypted_content
continuations failed with 400 invalid_encrypted_content and prefix
cache hits were lost.
The gateway boundary plugin now fills prompt_cache_key on outbound
openai_responses JSON bodies from the first non-empty of
x-claude-code-session-id, x-claude-session-id, or the inbound
metadata.user_id, and carries metadata.user_id onto the outbound body
when no metadata was set. A caller-supplied non-empty prompt_cache_key
always wins, and other protocols pass through untouched.
Fixes#1688
normalizeUsageInputTokens decides whether input_tokens already includes the
cached prefix, and it asked the upstream provider protocol first. Both usage
call sites merge the billing headers and the response body into one snapshot
and then normalize that merge once — but on a translated response the two
sources use different conventions:
- x-gateway-billing-* headers restate the upstream provider's own counters
verbatim, so for an OpenAI-compatible upstream they are cache-inclusive.
- The response body is whatever the gateway emitted. An Anthropic body is
cache-exclusive regardless of what it was translated from.
So an Anthropic response served from an OpenAI-compatible upstream had the
body's already-excluded prefix subtracted a second time. Non-streaming
responses were unaffected by luck: the headers win the merge and are
inclusive, making the subtraction correct. Streaming responses carry no
billing headers at all, leaving the body alone to be over-subtracted and then
clamped by Math.max(0, ...) — on a long cached conversation input_tokens
reports 0 on nearly every turn, and the derived cache ratio is pinned at
100%.
Tag each source with a UsageConventionSource and normalize the two separately
before merging them. Reordering the precedence instead is not sufficient: it
moves the defect onto the non-streaming path, which depends on the headers
being reduced.
Raw-trace updates carry no provider protocol, so their billing headers keep
falling back to the request path; only the body side changes there.
Four SSE response transforms decode every upstream chunk in isolation with
chunk.toString(). A UTF-8 character whose bytes straddle a chunk boundary is
therefore decoded twice as two invalid fragments, and each fragment becomes
U+FFFD. The character is destroyed before the block is ever parsed, so no
downstream code can recover it: the JSON still parses, the event still
validates, and the replacement characters are forwarded to the client.
The pending buffer these transforms already keep does not help. It joins
partial SSE *blocks*, not partial *characters* — by the time the bytes reach
it they have already been decoded and lost.
Affected sites, all in the response chain built in gateway/request/pipeline.ts:
codex-patch-bridge.ts transformSseChunk
codex-multi-agent-bridge.ts transformSseChunk
hosted-web-search/response-transform.ts
hostedWebSearchProtocolSseStream
anthropicHostedWebSearchProtocolSseStream
The correct pattern is already in this repository. anthropic-response-model.ts,
the last transform in that same chain, runs the identical block-splitting loop
over a node:string_decoder StringDecoder, which holds back an incomplete
trailing sequence until the next chunk supplies the rest.
Apply that pattern to the other four. Each transform now owns one decoder for
its lifetime, writes every chunk through it, and drains decoder.end() in flush
so a truncated stream still emits what the decoder was holding.
Any non-ASCII output is affected — CJK, Cyrillic, accented Latin, emoji — and
the apply_patch bridge carries file contents, so a corrupted character there is
written to disk.
Windows implements fsync as FlushFileBuffers, which behaves differently
from POSIX in two ways the raw trace spool relied on.
Flushing a directory handle is rejected outright with EPERM. syncDirectory
only tolerated EINVAL, ENOTSUP and EISDIR, so the durability barrier that
follows every rename threw instead of being skipped.
Flushing a file handle requires that handle to carry write access, and both
file flush sites opened their target read-only, so they also failed with
EPERM.
Together these made every durable write fail on Windows: with
observability.requestLogs enabled the sync handler answered 503
spool_unavailable for every bundle and nothing was ever persisted.
Tolerate EPERM alongside the other directory-flush rejections, and reopen
files as "r+" so their flush stays a real barrier rather than a tolerated
error. The tolerated set is extracted into a predicate so a test can pin
down which codes are skipped and, just as importantly, which ones still
propagate.
Fixes#1635
Address review on #1634:
- keep GrokRefreshAuthError module-private, consistent with the updated
Kimi implementation, avoiding new public API surface;
- dedupe concurrent resolveGrokAuth calls with a per-credential in-flight
refresh map keyed by sourceFile + authRecordKey, mirroring
kimiRefreshInFlight, so parallel requests share one refresh (and its
peer-rotation adoption outcome) instead of racing the same stale
refresh token;
- add a test proving concurrent refreshes coalesce into a single token
request.
Gabriel Tavares