Review follow-up: byModel is ai-gateway's normalized internal form
(parseModelScopedBody treats only `default` and actual model names as
scope keys); the raw config saved to disk is `default` plus top-level
model-name keys. The help text, i18n rows and the round-trip test now
use that raw shape.
The provider dialog had no field for either, so both were config-file
only: invisible in the UI and only editable by hand-writing the config
or writing to the config database directly.
Adds two JSON boxes to the Advanced settings section, next to the usage
connector body that already uses this pattern. They open pre-filled with
whatever the provider carries, and a malformed or non-object value blocks
the save with an inline error instead of writing something the gateway
would ignore.
extraBody and extraHeaders leave providerManualFieldsForSave now that the
draft round-trips them; billing, provider and transformer stay there.
Review follow-up: the spec hard-codes English UI strings ("Providers"
nav, the "Edit <name>" provider aria-label), but the web UI defaults to
the system language, so the suite failed on zh-locale hosts. The
Playwright context locale is now pinned per suite, which keeps the
selectors deterministic without dual-language lookups everywhere.
Drives the real management UI: seeds a provider carrying extraBody,
extraHeaders and a transformer, opens the edit dialog, saves, and asserts
the three fields are still there afterwards.
The spec fails on the unpatched build and passes with the fix, so it
guards the regression rather than the helper alone.
The CLI web server bootstrap moves out of cli-web.spec.ts into
cli-web-runtime.ts so both specs can start their own instance.
Saving a provider from the setup dialog replaced the stored entry with an
object rebuilt from the dialog draft, so every field the form does not
expose was dropped: extraBody, extraHeaders, transformer, billing and the
legacy provider protocol alias.
These only ever come from a hand-written config, and the loss is silent —
the provider keeps working, just without the upstream parameters it was
configured with. Re-typing a base URL on a provider carrying
`extraBody: { default: { reasoning_effort: "high" } }` is enough to turn
reasoning off with no error and no visible change in the UI.
Carry those fields over from the existing provider on edit. The legacy
apiKey/apikey/baseUrl/baseurl aliases are deliberately not carried over:
the form writes the canonical api_key/api_base_url, and a stale alias
would shadow the value the user just typed.
GatewayMediaExecutor.download() left the fetch Response body un-read and un-cancelled on several early-exit paths: a terminal non-ok HTTP status, an oversized declared content-length (declaredLength > maxApiArtifactBytes), and a mid-stream writeSync failure in the download loop. In each case the underlying undici connection/socket is leaked, so repeated failed downloads accumulate open connections and exhaust the pool.
Cancel the response body before throwing on those paths, and cancel the reader in the write-error handler, matching the cancellation already done on the retry branch and the in-loop size-limit branch.
Review follow-up (three defects):
1. imageBase64 accepted a full data:...;base64,... URL per its schema
documentation, but the new validation passed the whole string to the
base64 check and rejected it. Data URLs now take the same parsed path
as imageUrl via a shared dataUrlResult() helper.
2. Local files (imagePath/images[].path) were forwarded after wrapping
buffer bytes without inspecting them, so a .svg on disk went upstream
as a fake raster data URL and drew the exact strict-provider 400 this
work avoids. File bytes now go through the same content checks as every
other input.
3. The data URL media-type label was taken verbatim from the URL header or
the mimeType argument. Labels are now restricted to the supported raster
types (jpeg/png/gif/webp): an explicit supported label wins, otherwise
the format sniffed from the bytes, otherwise image/png.
Node >= 25 ships undici 8 as its built-in fetch while CCR bundles undici 7.
A dispatcher created by one undici major fails inside the other's fetch with
UND_ERR_INVALID_ARG ("invalid onError method"), so every gateway upstream
request fails with 502 "All target providers failed" (stage upstream_connect)
regardless of provider or proxy settings.
The generated gateway-proxy-preload.cjs already intercepts global fetch, but it
passed dispatcher-carrying calls straight to the runtime's built-in fetch and
injected bundled dispatchers into it. Route every call that carries or receives
a bundled dispatcher through the bundled undici's own fetch when the runtime
and bundled undici majors disagree; keep the previous behavior when they match.
Large (>=256KB) JSON request bodies were fully JSON.parse'd and pretty-printed
in the body formatter Web Worker, which then crashed (memory/CPU) on huge object
graphs - surfacing "Body formatter worker failed." and sticking at
"Loading body...". Preview truncation only applied to non-JSON bodies.
Now, in preview mode, an over-length text body is truncated to a plain-text
preview (createLogBodyPreviewText) instead of being parsed/pretty-printed, so the
worker never chokes. Full formatting still runs in "full" mode.
The judge is the real body.text length, not sizeBytes (which may be inflated
metadata and should still render its JSON tree when lightweight).
Fixes#1694
imageInputToUrl currently forwards whatever imageUrl/imageBase64 carries as
long as it is not an HTTP(S)/data URL, wrapping it in a blind
data:image/png;base64,... envelope. Strict upstreams answer malformed
payloads with a 400 that surfaces to the caller as a raw provider error.
Failure shapes seen in production:
- a local file path passed as imageUrl, or a bare [media_ref:...] id
- base64 truncated to length mod 4 == 1 (cut mid-image; padding cannot
restore it, the payload is dropped)
- an XML/SVG payload (typical vision upstreams accept only jpeg/png/gif/
webp, so relabeling it buys nothing)
- base64 truncated to length mod 4 == 2/3 (losslessly repairable by
adding = padding)
Anything that cannot be made into a well-formed data URL is now dropped
with a precise skip reason, and a call left with no usable image fails
with a message listing every reason instead of silently proceeding.
Requests converted to the OpenAI Responses protocol left the outbound
body without any session-stable field: prompt_cache_key was never set
and the inbound Anthropic metadata.user_id was dropped. Multi-channel
Responses upstreams that pin sessions on body fields hashed each turn
onto a different channel, so channel-bound encrypted_content
continuations failed with 400 invalid_encrypted_content and prefix
cache hits were lost.
The gateway boundary plugin now fills prompt_cache_key on outbound
openai_responses JSON bodies from the first non-empty of
x-claude-code-session-id, x-claude-session-id, or the inbound
metadata.user_id, and carries metadata.user_id onto the outbound body
when no metadata was set. A caller-supplied non-empty prompt_cache_key
always wins, and other protocols pass through untouched.
Fixes#1688
normalizeUsageInputTokens decides whether input_tokens already includes the
cached prefix, and it asked the upstream provider protocol first. Both usage
call sites merge the billing headers and the response body into one snapshot
and then normalize that merge once — but on a translated response the two
sources use different conventions:
- x-gateway-billing-* headers restate the upstream provider's own counters
verbatim, so for an OpenAI-compatible upstream they are cache-inclusive.
- The response body is whatever the gateway emitted. An Anthropic body is
cache-exclusive regardless of what it was translated from.
So an Anthropic response served from an OpenAI-compatible upstream had the
body's already-excluded prefix subtracted a second time. Non-streaming
responses were unaffected by luck: the headers win the merge and are
inclusive, making the subtraction correct. Streaming responses carry no
billing headers at all, leaving the body alone to be over-subtracted and then
clamped by Math.max(0, ...) — on a long cached conversation input_tokens
reports 0 on nearly every turn, and the derived cache ratio is pinned at
100%.
Tag each source with a UsageConventionSource and normalize the two separately
before merging them. Reordering the precedence instead is not sufficient: it
moves the defect onto the non-streaming path, which depends on the headers
being reduced.
Raw-trace updates carry no provider protocol, so their billing headers keep
falling back to the request path; only the body side changes there.
musi