GatewayMediaExecutor.download() left the fetch Response body un-read and un-cancelled on several early-exit paths: a terminal non-ok HTTP status, an oversized declared content-length (declaredLength > maxApiArtifactBytes), and a mid-stream writeSync failure in the download loop. In each case the underlying undici connection/socket is leaked, so repeated failed downloads accumulate open connections and exhaust the pool.
Cancel the response body before throwing on those paths, and cancel the reader in the write-error handler, matching the cancellation already done on the retry branch and the in-loop size-limit branch.
Node >= 25 ships undici 8 as its built-in fetch while CCR bundles undici 7.
A dispatcher created by one undici major fails inside the other's fetch with
UND_ERR_INVALID_ARG ("invalid onError method"), so every gateway upstream
request fails with 502 "All target providers failed" (stage upstream_connect)
regardless of provider or proxy settings.
The generated gateway-proxy-preload.cjs already intercepts global fetch, but it
passed dispatcher-carrying calls straight to the runtime's built-in fetch and
injected bundled dispatchers into it. Route every call that carries or receives
a bundled dispatcher through the bundled undici's own fetch when the runtime
and bundled undici majors disagree; keep the previous behavior when they match.
Requests converted to the OpenAI Responses protocol left the outbound
body without any session-stable field: prompt_cache_key was never set
and the inbound Anthropic metadata.user_id was dropped. Multi-channel
Responses upstreams that pin sessions on body fields hashed each turn
onto a different channel, so channel-bound encrypted_content
continuations failed with 400 invalid_encrypted_content and prefix
cache hits were lost.
The gateway boundary plugin now fills prompt_cache_key on outbound
openai_responses JSON bodies from the first non-empty of
x-claude-code-session-id, x-claude-session-id, or the inbound
metadata.user_id, and carries metadata.user_id onto the outbound body
when no metadata was set. A caller-supplied non-empty prompt_cache_key
always wins, and other protocols pass through untouched.
Fixes#1688
normalizeUsageInputTokens decides whether input_tokens already includes the
cached prefix, and it asked the upstream provider protocol first. Both usage
call sites merge the billing headers and the response body into one snapshot
and then normalize that merge once — but on a translated response the two
sources use different conventions:
- x-gateway-billing-* headers restate the upstream provider's own counters
verbatim, so for an OpenAI-compatible upstream they are cache-inclusive.
- The response body is whatever the gateway emitted. An Anthropic body is
cache-exclusive regardless of what it was translated from.
So an Anthropic response served from an OpenAI-compatible upstream had the
body's already-excluded prefix subtracted a second time. Non-streaming
responses were unaffected by luck: the headers win the merge and are
inclusive, making the subtraction correct. Streaming responses carry no
billing headers at all, leaving the body alone to be over-subtracted and then
clamped by Math.max(0, ...) — on a long cached conversation input_tokens
reports 0 on nearly every turn, and the derived cache ratio is pinned at
100%.
Tag each source with a UsageConventionSource and normalize the two separately
before merging them. Reordering the precedence instead is not sufficient: it
moves the defect onto the non-streaming path, which depends on the headers
being reduced.
Raw-trace updates carry no provider protocol, so their billing headers keep
falling back to the request path; only the body side changes there.
Four SSE response transforms decode every upstream chunk in isolation with
chunk.toString(). A UTF-8 character whose bytes straddle a chunk boundary is
therefore decoded twice as two invalid fragments, and each fragment becomes
U+FFFD. The character is destroyed before the block is ever parsed, so no
downstream code can recover it: the JSON still parses, the event still
validates, and the replacement characters are forwarded to the client.
The pending buffer these transforms already keep does not help. It joins
partial SSE *blocks*, not partial *characters* — by the time the bytes reach
it they have already been decoded and lost.
Affected sites, all in the response chain built in gateway/request/pipeline.ts:
codex-patch-bridge.ts transformSseChunk
codex-multi-agent-bridge.ts transformSseChunk
hosted-web-search/response-transform.ts
hostedWebSearchProtocolSseStream
anthropicHostedWebSearchProtocolSseStream
The correct pattern is already in this repository. anthropic-response-model.ts,
the last transform in that same chain, runs the identical block-splitting loop
over a node:string_decoder StringDecoder, which holds back an incomplete
trailing sequence until the next chunk supplies the rest.
Apply that pattern to the other four. Each transform now owns one decoder for
its lifetime, writes every chunk through it, and drains decoder.end() in flush
so a truncated stream still emits what the decoder was holding.
Any non-ASCII output is affected — CJK, Cyrillic, accented Latin, emoji — and
the apply_patch bridge carries file contents, so a corrupted character there is
written to disk.
musistudio