GatewayMediaExecutor.download() left the fetch Response body un-read and un-cancelled on several early-exit paths: a terminal non-ok HTTP status, an oversized declared content-length (declaredLength > maxApiArtifactBytes), and a mid-stream writeSync failure in the download loop. In each case the underlying undici connection/socket is leaked, so repeated failed downloads accumulate open connections and exhaust the pool.
Cancel the response body before throwing on those paths, and cancel the reader in the write-error handler, matching the cancellation already done on the retry branch and the in-loop size-limit branch.
Review follow-up (three defects):
1. imageBase64 accepted a full data:...;base64,... URL per its schema
documentation, but the new validation passed the whole string to the
base64 check and rejected it. Data URLs now take the same parsed path
as imageUrl via a shared dataUrlResult() helper.
2. Local files (imagePath/images[].path) were forwarded after wrapping
buffer bytes without inspecting them, so a .svg on disk went upstream
as a fake raster data URL and drew the exact strict-provider 400 this
work avoids. File bytes now go through the same content checks as every
other input.
3. The data URL media-type label was taken verbatim from the URL header or
the mimeType argument. Labels are now restricted to the supported raster
types (jpeg/png/gif/webp): an explicit supported label wins, otherwise
the format sniffed from the bytes, otherwise image/png.
imageInputToUrl currently forwards whatever imageUrl/imageBase64 carries as
long as it is not an HTTP(S)/data URL, wrapping it in a blind
data:image/png;base64,... envelope. Strict upstreams answer malformed
payloads with a 400 that surfaces to the caller as a raw provider error.
Failure shapes seen in production:
- a local file path passed as imageUrl, or a bare [media_ref:...] id
- base64 truncated to length mod 4 == 1 (cut mid-image; padding cannot
restore it, the payload is dropped)
- an XML/SVG payload (typical vision upstreams accept only jpeg/png/gif/
webp, so relabeling it buys nothing)
- base64 truncated to length mod 4 == 2/3 (losslessly repairable by
adding = padding)
Anything that cannot be made into a well-formed data URL is now dropped
with a precise skip reason, and a call left with no usable image fails
with a message listing every reason instead of silently proceeding.
Requests converted to the OpenAI Responses protocol left the outbound
body without any session-stable field: prompt_cache_key was never set
and the inbound Anthropic metadata.user_id was dropped. Multi-channel
Responses upstreams that pin sessions on body fields hashed each turn
onto a different channel, so channel-bound encrypted_content
continuations failed with 400 invalid_encrypted_content and prefix
cache hits were lost.
The gateway boundary plugin now fills prompt_cache_key on outbound
openai_responses JSON bodies from the first non-empty of
x-claude-code-session-id, x-claude-session-id, or the inbound
metadata.user_id, and carries metadata.user_id onto the outbound body
when no metadata was set. A caller-supplied non-empty prompt_cache_key
always wins, and other protocols pass through untouched.
Fixes#1688
normalizeUsageInputTokens decides whether input_tokens already includes the
cached prefix, and it asked the upstream provider protocol first. Both usage
call sites merge the billing headers and the response body into one snapshot
and then normalize that merge once — but on a translated response the two
sources use different conventions:
- x-gateway-billing-* headers restate the upstream provider's own counters
verbatim, so for an OpenAI-compatible upstream they are cache-inclusive.
- The response body is whatever the gateway emitted. An Anthropic body is
cache-exclusive regardless of what it was translated from.
So an Anthropic response served from an OpenAI-compatible upstream had the
body's already-excluded prefix subtracted a second time. Non-streaming
responses were unaffected by luck: the headers win the merge and are
inclusive, making the subtraction correct. Streaming responses carry no
billing headers at all, leaving the body alone to be over-subtracted and then
clamped by Math.max(0, ...) — on a long cached conversation input_tokens
reports 0 on nearly every turn, and the derived cache ratio is pinned at
100%.
Tag each source with a UsageConventionSource and normalize the two separately
before merging them. Reordering the precedence instead is not sufficient: it
moves the defect onto the non-streaming path, which depends on the headers
being reduced.
Raw-trace updates carry no provider protocol, so their billing headers keep
falling back to the request path; only the body side changes there.
Four SSE response transforms decode every upstream chunk in isolation with
chunk.toString(). A UTF-8 character whose bytes straddle a chunk boundary is
therefore decoded twice as two invalid fragments, and each fragment becomes
U+FFFD. The character is destroyed before the block is ever parsed, so no
downstream code can recover it: the JSON still parses, the event still
validates, and the replacement characters are forwarded to the client.
The pending buffer these transforms already keep does not help. It joins
partial SSE *blocks*, not partial *characters* — by the time the bytes reach
it they have already been decoded and lost.
Affected sites, all in the response chain built in gateway/request/pipeline.ts:
codex-patch-bridge.ts transformSseChunk
codex-multi-agent-bridge.ts transformSseChunk
hosted-web-search/response-transform.ts
hostedWebSearchProtocolSseStream
anthropicHostedWebSearchProtocolSseStream
The correct pattern is already in this repository. anthropic-response-model.ts,
the last transform in that same chain, runs the identical block-splitting loop
over a node:string_decoder StringDecoder, which holds back an incomplete
trailing sequence until the next chunk supplies the rest.
Apply that pattern to the other four. Each transform now owns one decoder for
its lifetime, writes every chunk through it, and drains decoder.end() in flush
so a truncated stream still emits what the decoder was holding.
Any non-ASCII output is affected — CJK, Cyrillic, accented Latin, emoji — and
the apply_patch bridge carries file contents, so a corrupted character there is
written to disk.
Gabriel Tavares