## Summary
`httpapi.Read` decoded request bodies with no size limit, so a single
request could allocate memory without bound. This adds a 4 MiB default
ceiling, leaves the endpoints that legitimately need more explicitly
exempted, and counts the rejections so a limit set too tight is visible.
This is the first of three PRs split out of #28048, covering the
endpoints that answer in `codersdk.Response` shape. The OAuth2 decode
paths (RFC 6749, RFC 7591) and the SCIM ones (RFC 7644) answer in their
own error shapes and follow in separate PRs, along with the lint rule
that pins the invariant.
Closes PLAT-463. Remediates SEC-416 (CWE-770, CVSS 7.5) and SEC-392.
## Problem
`httpapi.Read` calls `json.NewDecoder(r.Body).Decode(value)` with no
ceiling, and no middleware in the chain bounds body size. The exposure
is pre-authentication: login, OTP, and first-user creation all read a
body before any authorization decision is reached. The existing rate
limiter bounds request *rate*, which is orthogonal to the memory a
single admitted request may consume.
## Fix
`Read` is split into `Read` and `ReadLimit`. `ReadLimit` wraps `r.Body`
in an `http.MaxBytesReader` and keeps the existing decode and validate
logic; `Read` delegates to it with a new `DefaultMaxRequestBodyBytes` of
4 MiB, which covers the 124 remaining non-test callers at a single site.
`http.MaxBytesReader` composes as tightest-wins, so the handlers that
pre-wrapped their own bodies pass their limit to `ReadLimit` rather than
wrapping, and each keeps its previous ceiling byte for byte. That
matters most for the bulk secrets import at `8 * MaxSecretsFileBytes`:
an unconditional wrap inside `Read` would have silently halved it to the
default. `TestImportUserSecretsBodyLargerThanDefaultLimit` is the
regression guard for that specific failure, and
`TestMaxBytesReaderNesting` pins the composition behavior the whole
requirement rests on.
Every rejection site calls `httpapi.RecordRequestBodyLimit`, which names
the limit that tripped on the request's existing log line and marks the
request so `coderd_api_requests_too_large_total{reason="request_body"}`
counts body rejections apart from the 413s coderd answers for other
causes, such as agent log storage overflow. A limit set too tight for a
legitimate payload therefore surfaces without waiting for a user report.
The limit is a constant rather than a deployment option: an operator
raising it to unblock something would reopen the vulnerability as
configuration, where a security scan will not find it. A legitimate 413
is answered with a targeted `ReadLimit` on that endpoint.
## Behavior change
`POST /api/v2/files` now answers 413 rather than 400 when a request body
exceeds `HTTPFileMaxBytes`. It installed that bound already but reported
the rejection as a read failure, which leaked the stdlib `http: request
body too large` string through `Detail` and kept the largest limit in
the tree off the metric. The separate 413 for an oversized expanded
archive is unchanged.
The task log snapshot endpoint now answers 413 rather than 400 when its
64 KiB cap is exceeded. Routing it through `ReadLimit` also changes its
decode-failure message from "Failed to decode request payload." to
"Request body must be valid JSON.", which is what every other endpoint
answers. Its tests are updated to match both.
`coderd_api_requests_too_large_total` is new, so there is no existing
query to migrate. It counts the 413s coderd answers, labeled `method`,
`path`, and `reason`. `reason="request_body"` is a rejection by one of
the limits above; `reason="other"` is a 413 that has nothing to do with
body size, such as agent log storage overflow.
## Reading this
The commits are ordered to be read in sequence. Commits 1 and 2 are the
security fix; commits 3 to 5 are the observability consequences, and
commit 3 is the one that touches dashboards. Commit 7 documents the
limit on the REST API reference index. Commits 6 and 8 add and revert an
exhaustive `@Failure 413` annotation pass, which buried the fix under
its regenerated swagger, and cancel out.
The cardinality notes in `aibridge/metrics/metrics.go` assume the
`provider` label takes one of three values, and two for the key pool
metrics. That was accurate when the notes were written: `provider` is
the provider instance name, and the name defaulted to one of the three
provider types aibridge supports. Instances can now be given their own
names, so the label takes any configured name and the series counts
scale with the number of configured providers rather than being capped
at a fixed number.
The monitoring docs are also updated to make clear that `provider` is
the provider instance name.
Comments and documentation only, no behaviour change.
Follow-up to #28210.
## Stack Context
PR 5 of 6 in a stack that migrates every Coder MCP surface from the
archived `github.com/mark3labs/mcp-go` library to the official
`github.com/modelcontextprotocol/go-sdk` v1.7.0.
Stack: #28056 -> #28057 -> #28058 -> #28059 -> #28060 -> #28061
## Why
The aibridge injected-MCP proxy now owns an official `*mcp.Client`,
`*mcp.StreamableClientTransport`, and `*mcp.ClientSession`.
- The proxy constructor accepts an optional `*http.Client` instead of
mark3labs options; the header-injecting wrapper shallow-copies a
supplied client so its Timeout, Jar, and redirect policy survive.
- Manual protocol version negotiation and the mark3labs five-second
close workaround are removed; the SDK negotiates during `Connect` and
fails when no mutually supported version exists.
- Repeated `Init` closes the previous session, and a failed tool fetch
closes the just-created session so transports do not leak.
- Tool and intercept types use the official pointer content types;
embedded resource blobs are re-encoded to base64 for model-facing text
because the SDK decodes them into raw bytes.
- `aibridge/mcpmock` is regenerated, and its stale `go:generate` source
path is corrected.
> Mux created this PR on Mike's behalf.
_Disclosure: investigated and drafted with Claude Opus 5. I reviewed the
change and ran the tests locally._
Streaming Responses interceptions recorded no token usage when the
bridge was built with a nil `mcp.ServerProxier`, because
`recordTokenUsage` was called from inside the `i.mcpProxy != nil` branch
in `aibridge/intercept/responses/streaming.go`. Requests completed
normally and returned `200`, so traffic was served but metered as zero,
with no error surfaced. Upstream reports usage on the
`response.completed` event independently of tool injection, so the
proxier is not a valid precondition for recording it.
That state is reachable in production:
`coderd/aibridged/pool.go:255-265` treats proxier construction failure
as non-fatal ("MCP server injection can gracefully degrade") and caches
the resulting bridge via `SetWithTTL`, so one transient config-retrieval
error suppressed usage recording for every streaming Responses request
served by that bridge until its TTL expired.
This records usage for every completed response, guarded only on
`completedResponse`, matching `responses/blocking.go` and both
`chatcompletions` implementations. Per-iteration semantics are preserved
for the inner agentic loop.
The integration harness substituted a non-nil noop manager whenever no
proxier was supplied (`setupbridge.go:153-155`), so the nil path was
never exercised. `withoutMCP()` covers it.
Verification, with the fix reverted:
```
--- FAIL: TestResponsesStreamingRecordsTokenUsageWithoutMCP/without_mcp_proxy
Error: "[]" should have 1 item(s), but has 0
--- PASS: TestResponsesStreamingRecordsTokenUsageWithoutMCP/with_noop_mcp_proxy
```
and with it applied:
```
--- PASS: TestResponsesStreamingRecordsTokenUsageWithoutMCP/without_mcp_proxy
--- PASS: TestResponsesStreamingRecordsTokenUsageWithoutMCP/with_noop_mcp_proxy
--- PASS: TestResponsesStreamingRecordsTokenUsagePerAgenticIteration
```
The per-iteration case asserts exactly two records for the injected-tool
fixture, so decoupling the call from the proxier does not double-count
when the agentic loop iterates. `go test -race ./aibridge/...` passes
across all 15 packages.
Fixes#27885
One caveat on verification: I was unable to run `make gen` / `make
pre-commit` locally, as I do not have the full mise toolchain installed.
The change touches no codegen inputs (no SQL, protos, mocks, or
TypeScript), so I do not expect generated-file drift, but flagging it
rather than leaving it implied.
Our fantasy fork had drifted far behind upstream charmbracelet/fantasy
(base v0.31.0 vs current v0.40.0). This PR updates the pinned forks
after reconciling which fork hacks upstream has fixed and which we still
need, and adapts this repo to the new APIs.
## Fork updates
- `charm.land/fantasy` ->
[coder/fantasy#51](https://github.com/coder/fantasy/pull/51) (merged):
`coder_2_33` synced with upstream v0.40.0, pinned at the merge commit
`bb10946892ef`.
- `github.com/openai/openai-go/v3` ->
[coder/openai-go#10](https://github.com/coder/openai-go/pull/10)
(merged): `coder/pinned` rebased from v3.16.0 onto upstream v3.50.0
(required by upstream fantasy), pinned at the merge commit
`92b5addb22d2`.
- `coder/anthropic-sdk-go` pin unchanged; the fantasy fork now tracks
the same revision this repo ships.
## Hack reconciliation summary
Dropped from our fantasy diff (upstream now has equivalents, often
stricter): truncated-stream fail-closed detection, Anthropic EffortXHigh
/ computer use / thinking effort / thinking display, replay fidelity for
signed reasoning and web_search errors, PDF and text documents with
sanitized filename titles, refusal finish-reason mapping (upstream also
maps Bedrock `content_filtered`/`guardrail_intervened`), gpt-5.5/5.6
Responses routing, the Go 1.25 downgrade, and the openai-go SSE decoder
and appendCompact patches.
Still fork-only and preserved: OpenAI computer use, OpenAI Responses
replay continuity validation, Anthropic pre-4.6 budget-thinking
conversion plus explicit thinking disable for effort none, Anthropic
RefusalMetadata parsing, Bedrock cross-region inference profile region
mirroring, and openai-go deferred body serialization with the
WithJSONSet fix.
Picked up new upstream features: stream transport retry with in-band SSE
error classification, Bedrock expired-credential refresh, per-message
cache markers for OpenAI-compatible models, tool panic recovery, extra
usage fields in provider metadata, and ClientMetadata on tool results.
## Changes in this repo
- `aibridge/intercept/responses`: `ResponseOutputItemUnion.Arguments`
became a union type in openai-go v3.50; read function-call arguments via
`.OfString` (plus test literal updates).
- `coderd/x/chatd/chatdebug`: register the new fantasy `Call.Headers`,
`ObjectCall.Headers`, and `ToolResultPart.ClientMetadata` fields in the
normalization coverage map (all skipped).
- `aibridge/internal/integrationtest`: make the RST test listener drain
the request before resetting the connection. The new SDK's write path
exposed the previous 1-byte-read race as sporadic `use of closed network
connection` failures; the fix holds over 40 consecutive runs.
- `go.mod`: rewrite the fork provenance comments to describe the
post-sync state.
## Validation
- `go build ./...` and `go vet ./...` clean (vet findings identical to
base).
- Fresh (`-count=1`) runs of `./coderd/x/chatd/...`, `./aibridge/...`,
`./coderd/aibridged/...`, `./coderd/database/db2sdk/`: 37 packages pass.
- `TestClientAndConnectionError` stress-tested 40x clean.
- Both fork PRs have green CI.
> Mux acted on Mike's behalf to create this PR.
## Problem
CODAGT-906: chats using OpenAI-compatible backends (e.g. poolside)
through the AI Bridge persist token usage inflated 105x-640x, which
falsely triggers automatic chat compaction on every turn.
The chat-completions streaming interceptor summed usage across every SSE
chunk of one upstream stream and rewrote each relayed usage-bearing
chunk with that running sum. Spec-compliant OpenAI emits usage once
(final chunk with `stream_options.include_usage`), so the sum equals the
final value. vLLM-style backends emit cumulative usage snapshots on
every chunk, so the relayed final usage becomes roughly `N_chunks x
prompt_tokens` (e.g. 417,012 persisted for a ~6,000-token context).
chatd persists that value per assistant message and its compaction
trigger reads it as context occupancy.
## Fix
Track the latest usage-bearing chunk's raw usage (last-wins) in the
stream processor, updating only when a chunk actually carries usage so a
trailing usage-less chunk cannot zero it. `marshalChunk` relays that
value and `recordTokenUsage` records the same value, unifying relayed
and recorded usage. Last-wins is correct for both shapes: a single final
usage chunk, and cumulative snapshots where each snapshot already
includes all prior tokens.
Per-iteration semantics are unchanged: each tool-loop iteration has its
own processor, and the final iteration's usage is what the client sees.
## Tests
- `TestStreamProcessorUsage` (internal): cumulative snapshots with a
trailing usage-less chunk, and the spec-compliant final-only shape;
asserts relayed and recorded usage equal the last snapshot.
- New txtar fixture `streaming_cumulative_usage_injected_tool.txtar`
with per-chunk cumulative usage plus an injected tool call; asserts
client-visible final usage through the full interceptor.
- Red-green verified: with the fix reverted, the internal test reports
zeroed usage (trailing chunk overwrite) and the fixture test reports
18000 summed prompt tokens instead of 6000.
The blocking (non-streaming) path deliberately keeps its cross-iteration
summation for external clients and is untouched. Remote dogfood UAT
validated chat streaming, tool calls, plausible usage numbers, and zero
spurious compactions.
> Mux acted on Mike's behalf to author this change.
Adds client context to `Client()` method in `aibridged.Server`,
effectivly renaming `ClientContext()` method as `Client()`.
Similarly `aibridged.ClientFuncWithContext` became
`aibridged.ClientFunc`.
`aibridged.Server.Client()` acquired a DRPC client with
`context.Background()`, callers in theory could wait indefinitely for
the daemon to connect to coderd.
Every call site already had a context except the recorder callback.
`aibridge.NewRecorder` takes a `func(context.Context) (Recorder, error)`
and acquires against the record call's context.
Adds sonnet 5 to the list of models that require adaptive thinking for
Bedrock InvokeModel.
Smoke-tested locally.
> Obligatory disclosure: a Coder agent helped with this.
## Problem
When a key returned 401 or 403, the pool marked it permanently
unavailable for the lifetime of that in-memory pool. This is bad UX: a
transient auth failure or a briefly-misconfigured key could take a key
out of rotation until the operator either restarted Coder or
reconfigured the key (even re-saving the same working value).
## Changes
- **403 removed from key failover**: it's a per-request authorization
failure, not a key-level problem, so it's surfaced to the caller as-is
without marking the key or failing over.
- **401 now applies a temporary cooldown** (like 429) so the key
recovers on its own instead of staying blocked.
- When every key is in an auth-failure cooldown, the pool reports a
`502` with no `Retry-After`, but the keys still recover automatically
once the cooldown elapses.
Closes
https://linear.app/codercom/issue/AIGOV-421/ai-gateway-a-quarantined-centralized-key-never-recovers-without-a
Closes
https://linear.app/codercom/issue/AIGOV-533/403s-misclassifying-keys-as-permanently-down-in-ai-gateway
> [!NOTE]
> Initially generated by Claude Opus 4.7, modified and reviewed by
@ssncferreira
Implements
https://linear.app/codercom/issue/AIGOV-213/add-bedrock-provider
# AWS Bedrock mantle support in AI Gateway
## Summary
Add support for the AWS Bedrock **mantle** endpoint
(`bedrock-mantle.{region}.api.aws/anthropic/v1/messages`) to AI Gateway.
Mantle serves Claude through the native Anthropic Messages API. We model
it as a `protocol` field on the existing Bedrock provider settings
(`invoke-model` default, or `mantle`) rather than as a new provider
type, and we treat mantle as a pure passthrough: SigV4-sign and forward,
no body translation.
## Background
Claude on AWS Bedrock is reachable through two endpoints, each speaking
exactly one wire protocol:
1. **InvokeModel** (existing): `bedrock-runtime.{region}.amazonaws.com`.
Model id in the URL path, request translated into Bedrock's InvokeModel
format, responses returned as a binary AWS eventstream. This is what AI
Gateway already supported for Bedrock.
2. **Mantle** (this doc):
`bedrock-mantle.{region}.api.aws/anthropic/v1/messages`. Native
Anthropic Messages API: model in the body, plain SSE streaming.
## Why a `protocol` field, not a new provider type
The alternative is to model mantle as its own `ai_provider_type`
(`bedrock-mantle`) alongside `bedrock`. I chose the `protocol` field
instead for two reasons:
1. Mantle reads more like a protocol of Bedrock than a separate
provider. It is the same AWS account, credentials, region, and IAM,
reached over a different wire protocol and host. One Bedrock provider
with two protocols (`invoke-model` default and `mantle`) models that
more organically than two provider types.
2. It avoids a database migration. The `protocol` field lives in the
settings JSON blob (empty resolves to `invoke-model`, so existing
providers are unaffected), whereas a new type means an enum value and
the `ALTER TYPE ... ADD VALUE` migration that goes with it.
## Why passthrough, not translation
The client already emits Bedrock-legal requests in mantle mode:
```sh
export CLAUDE_CODE_USE_MANTLE=1
export CLAUDE_CODE_SKIP_MANTLE_AUTH=1
export ANTHROPIC_BEDROCK_MANTLE_BASE_URL=https://<coder>/api/v2/aibridge/<provider-name>
```
So the gateway just forwards the body and SigV4-signs it (service
`bedrock-mantle`), and skips all the InvokeModel body-translation (model
remap, thinking conversion, beta-flag allowlist, field stripping). This
keeps the mantle path thin and avoids a second copy of translation logic
to maintain.
## Consequences
- Protocol-dependent fields: `model` / `small_fast_model` are used by
InvokeModel but ignored by mantle (the client sends the model), and
`base_url` is required for mantle but optional for InvokeModel.
Validation is protocol-aware.
- No central model control on mantle: because it is a passthrough, the
operator cannot pin the model.
- `region` and the `base_url` host must name the same region (the SigV4
scope must match the endpoint); a mismatch surfaces as `Credential
should be scoped to a valid region`.
## Draft UI
<img width="1100" height="579" alt="image"
src="https://github.com/user-attachments/assets/37bab46d-8958-4a96-9f47-1fef3493e1b6"
/>
## Follow-up PRs:
- https://github.com/coder/coder/pull/27156
Categorises the terminal error of a failed interception and persists it
on the interception record, then surfaces it on the AI Gateway API.
- Categorise into an enum (`bad_request`, `unauthorized`,
`rate_limited`, `overloaded`, `server_error`, `unknown`), unwrapping
the ResponseError envelope, the upstream Anthropic/OpenAI SDK errors,
and key-pool exhaustion so blocking and streaming paths agree.
- Thread the type and raw message through the recorder dRPC into the
`aibridge_interceptions` row (optional proto fields; NULL on success).
- Expose the error on the AI Gateway thread API from the root
interception.
*This PR was produced by opencode (agent) using the `anthropic/claude-opus-4-8` model, under human direction and review.*
## Summary
The Responses interceptor previously recorded only `function_call` and `custom_tool_call` output items, so interceptions that did real work via built-in tools (`web_search_call`, `computer_call`, `shell_call`, `mcp_call`, etc.) recorded no tool usage at all.
`recordNonInjectedToolUsage` now whitelists every tool-call output type and records it, with the tool name falling back to the item type when none is set.
`ToolUsageRecord` also gains an `ItemID` field so the two distinct Responses identifiers are captured without conflation (addresses review feedback on coder/aibridge#273):
- `ItemID`: the output item's unique `id` (always present).
- `ToolCallID`: the `call_id` correlation id (empty for hosted tools the provider runs server-side).
## Tests
- Extends `TestRecordToolUsage` with cases for the new hosted/agentic tool types.
- Adds blocking and streaming `web_search` fixtures (scrubbed of credentials and identifying metadata) plus `TestResponsesOutputMatchesUpstream` cases asserting a hosted tool records with an empty `ToolCallID` and a populated `ItemID`.
Linear: AIGOV-96
---
*This PR was produced by opencode (agent) using the `anthropic/claude-opus-4-8` model, under human direction and review.*
A Bedrock provider that assumes an IAM role kept failing with
`AssumeRole` `AccessDenied` for several minutes after its target role's
trust policy was changed, and only recovered on a gateway restart or a
long wait. The request itself was correct: the AWS CLI, using the same
identity and the same `ExternalId`/role/region, accepted the identical
request immediately against the same endpoint.
The difference is the connection. The Go SDK reuses a keep-alive
connection for the STS client, so every `AssumeRole` rides one
connection pinned to a single STS endpoint. After a trust-policy change,
that connection kept returning `AccessDenied` for minutes while a fresh
connection (the AWS CLI) accepted the identical request at once; it
recovered only when the connection recycled or the process restarted.
The exact STS-internal reason is unconfirmed (likely per-endpoint
propagation of the change) — what is verified is that a fresh connection
per call recovers promptly.
Disable keep-alive on the STS client so each `AssumeRole` opens a fresh
connection and a trust-policy update takes effect quickly. `AssumeRole`
runs at most once per credential-cache lifetime, so keep-alive bought
nothing here. The change is scoped to the STS client only; Bedrock model
requests are signed by a separate client and keep their connection
pooling.
## What the data proves
| | CLI | Gateway |
|--------------------|-------------------------------------------|------------------------------------------|
| Identity / key | `bedrock-base-user-useless` / `AKIA…44NL` | same |
| STS endpoint | `sts.us-east-2.amazonaws.com` | same |
| Request params | `ExternalId=QL53…`, role, session, 900 | same |
| Recovery after fix | 7 seconds (21:27:54) | ~4.5 minutes (21:32:17) |
| Re-hitting AWS? | new call each time | yes — 77 fresh `AssumeRole`s,
all denied |
Same identity, params, and endpoint, concurrent — yet the gateway was
denied for ~4.5 minutes while the CLI recovered in 7 seconds, and the
gateway made a fresh `AssumeRole` on every request (so it was not
caching a failure). The only difference was connection reuse.
After disabling keep-alive, the same break/fix experiment brought
gateway recovery down from ~4.5 minutes to ~7 seconds, in lockstep with
the AWS CLI.
Implements:
https://linear.app/codercom/issue/AIGOV-495/add-externalid-to-prevent-confused-deputy-problem
When a Bedrock provider assumes an IAM role via STS, the gateway now
generates a unique external ID for it and sends that value on every
`AssumeRole` call. The external ID guards against the [confused deputy
problem](https://docs.aws.amazon.com/IAM/latest/UserGuide/confused-deputy.html)
on cross-account role assumption. Per [AWS's
recommendation](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-user_externalid.html),
the gateway generates and owns the value rather than accepting one from
the operator; that ownership is what makes it effective, since a party
who knows another's external ID can't induce the gateway to send it.
The external ID is server-owned and read-only over the API. It is
generated once, when a provider first has a `role_arn`, and is stable
thereafter. Clients cannot set it: create rejects any supplied
`external_id`, and update rejects a value that differs from the stored
one. An update may echo the stored value back unchanged, so the normal
read-modify-write flow (GET the provider, change a field, PATCH the full
settings object) keeps working. The value is not a secret and is
returned on GET so operators can copy it into the target role's trust
policy as an `sts:ExternalId` condition.
It is persisted in the existing JSON settings blob, so there is no
migration or audit-table change.
Implements
[AIGOV-481](https://linear.app/codercom/issue/AIGOV-481/featai-gateway-support-anthropic-v1messages-route-on-the-copilot).
Adds the Anthropic-style `/v1/messages` route to the Copilot provider.
GitHub Copilot CLI (1.0.65) using the default `Claude Sonnet 4.6
(default)` model sends Anthropic-style `/v1/messages` requests through
`aibridgeproxyd` to the Copilot provider. The provider only registered
the OpenAI-compatible `/chat/completions` and `/responses` routes, so
these requests failed:
```
CAPIError: 404 404 404 route not supported: POST /copilot/v1/messages
```
*This PR was produced by opencode (agent) using the
`anthropic/claude-opus-4-8` model, under human direction and review.*
Wire the Agent Firewall correlation headers
(`X-Coder-Agent-Firewall-Session-Id` and
`X-Coder-Agent-Firewall-Sequence-Number`) through the AI Bridge
interception processor so that each interception is linked to its
originating firewall session.
Closes https://linear.app/codercom/issue/AIGOV-259
> Generated by Coder Agents on behalf of @SasSwart
**Data flow:**
`request header` → `bridge.go` reads + strips → `InterceptionRecord` →
`translator.go` → proto `RecordInterceptionRequest` →
`aibridgedserver.go` → DB
When adding chatd tests to route through a real in-process `aibridged`
daemon (#26658), found two races:
- **Pool:** `CachedBridgePool.Shutdown` calls `cache.Close()` while an
in-flight `Acquire` runs `cache.Wait()`. ristretto closes the channel
`Wait` sends on.
- **Bridge:** `RequestBridge.ServeHTTP` does `inflightWG.Add(1)` after
the `b.closed` check, racing `Shutdown`'s `inflightWG.Wait()`.
## Fix
- `RequestBridge`: adds `admitMu` RWMutex to order `inflightWG.Add`
(ServeHTTP, read) before `close(b.closed)` (Shutdown, write).
- `CachedBridgePool`: adds `opsMu` + `opsWG` so `Shutdown` drains
in-flight `Acquire`/`ReplaceProviders` before `cache.Close()`
- Adds tests `TestRequestBridgeShutdownAdmissionRace` and
`TestPoolShutdownReplaceProviders` for above. (Note:
`TestRequestBridgeShutdownAdmissionRace` leverages a `serve_admission`
quartz trap added to `RequestBridge`).
---
> 🤖 Created by Coder Agents on behalf of @johnstcn.
We currently have two possible sources of truth for the Bedrock region:
* `cfg.Region`, when explicitly provided (this also covers the case
where the UI parses the base URL and populates `cfg.Region`)
* the region resolved from the AWS environment
An explicitly configured region should always take precedence over the
environment-derived region.
I suggest implementing this resolution policy in the `NewAnthropic`
constructor so that, after initialization, there is a single source of
truth for the resolved region.
Migrate `wg.Add(1); go func() { defer wg.Done(); ... }()` to
`wg.Go(func() { ... })` in tests.
Where the prior pattern passed the loop variable explicitly via a
closure parameter (`go func(id int) { ... }(i)`), drop the parameter and
reference the loop variable directly. Per-iteration loop variables since
Go 1.22 make this safe.
## Description
Updates comments, error strings, and documentation within the
`aibridge/` package to use the new AI Gateway naming, following the
backend rename in #26475.
## Changes
- Update `aibridge/provider/provider.go` comment examples from
`/aibridge` to `/ai-gateway` and "AI Bridge" to "AI Gateway"
- Update `aibridge/AGENTS.md` architecture description
- Update `aibridge/README.md` mount path examples from
`/api/v2/aibridge/` to `/api/v2/ai-gateway/`
- Rename "AI Bridge" to "AI Gateway" in comments across `bridge.go`,
`intercept/client_headers.go`, `intercept/responses/base.go`,
`intercept/messages/base.go`, and `intercept/messages/reqpayload.go`
- Update error string in `intercept/responses/base.go` and matching test
assertion
Addresses
https://github.com/coder/coder/pull/26475#pullrequestreview-4544441981
Refs https://linear.app/codercom/issue/AIGOV-226
> Generated with the assistance of Coder Agents (@ssncferreira)
Bedrock rejects legacy `thinking.type=enabled` requests for Claude Opus
4.8 because the model requires adaptive thinking. The AI Bridge Bedrock
shim only recognized Opus 4.7 as adaptive-only, so Opus 4.8 requests
could fall through and produce Bedrock 400 responses.
Add Opus 4.8 to the adaptive-only model detection and cover the regional
Bedrock model ID form with a regression test.
<details>
<summary>Coder Agents disclosure</summary>
This PR was generated by Coder Agents on behalf of @ericpaulsen.
</details>
# Support IAM role assumption for AWS Bedrock in AI Bridge
## Summary
Implements
https://linear.app/codercom/issue/AIGOV-371/support-dynamic-bedrock-assumerole-across-aws-accounts-for-ai-gateway
A Bedrock provider can now be configured with an IAM role to assume.
Before calling Bedrock, the gateway assumes that role via STS and signs
requests with the resulting temporary credentials. Whether the role
lives in the same account or another one is entirely a matter of the
role's trust policy.
## Problem
Many organizations prohibit long-lived AWS access keys and expect
workloads to authenticate through assumed IAM roles instead. A common
case is an organization that runs Bedrock across several AWS accounts,
one per business unit, and needs each unit's usage billed to its own
account by assuming a role there. AI Bridge previously authenticated a
Bedrock provider only with static keys or the gateway's own ambient AWS
identity, which is shared by every provider, with no way to assume a
role. These deployments had no clean path.
## How it works
When a provider is configured with a role ARN, the gateway uses its base
identity to assume that role via STS and signs Bedrock requests with the
temporary credentials it returns. The base identity is whatever the AWS
default credential chain resolves, IRSA, EKS Pod Identity, EC2 Instance
Profile, or static keys.
Credentials are resolved once when the provider is set up and are then
cached and rotated, so individual requests are served from the cache
rather than triggering a new STS call. A deployment that needs several
roles configures several providers, each pointing at its own role.
## Configuration
The role ARN is part of the Bedrock provider settings and is set through
the AI provider API. It is optional: a provider with no role ARN behaves
exactly as before.
## Scope and trade-offs
- This PR is backend only. The settings UI for the role ARN ships in a
follow-up.
- Configuration is not exposed through environment variables.
Environment-based provider configuration is being phased out in favor of
database-managed providers, so the role ARN is intentionally database
and API only.
Follow-up PR: https://github.com/coder/coder/pull/26578
In our codebase we have an existing convention of using
`flag.Lookup("test.v")` instead of `testing.Testing()`. This avoids
pulling in the entire `testing` package. Another consequence: some of
our custom linters trigger upon import of the `testing` package which
can lead to unexpected linter errors.
PRs https://github.com/coder/coder/pull/26092 and
https://github.com/coder/coder/pull/26519 both landed on main and the
circuit breaker test file ended up importing both
`aibridge/internal/testutil` and `coder/v2/testutil` under the same
name, causing a redeclaration error and breaking `make lint`.
Alias the latter as `codertestutil`, matching the convention already
used in `passthrough_internal_test.go` and `keyfailover_test.go`.
`TestCircuitBreaker_FullRecoveryCycle/OpenAI` flaked once on macOS CI.
The most likely cause is that the circuit breaker `Timeout`
(open-to-half-open transition) was too short relative to the time
between test phases. On a slow runner, the breaker could transition to
half-open before the test verified it was still open, so the request
went through as a half-open probe instead of being rejected.
Increases `Timeout` to `testutil.IntervalMedium` (250ms) across all
circuit breaker integration tests.
**Note:** Ideally, these tests would use a mock clock for deterministic
timing, but https://github.com/sony/gobreaker (the library used for
circuit breaker logic) uses real time internally and doesn't expose a
clock interface.
Closes https://linear.app/codercom/issue/AIGOV-438
> Generated with [Coder Agents](https://coder.com/agents) on behalf of
@ssncferreira
Applies follow-ups from the key pool failover work:
- Add a test verifying key pool state is shared across bridged and passthrough routes.
- Refactor the key failover and passthrough tests to use the shared `MockUpstream` helper.
- Simplify how the request body option is passed through the Anthropic messages interceptor.
- Make `ResponseErrorFromKeyPool` nil-safe and cover it with a test.
Closes: https://linear.app/codercom/issue/AIGOV-398/small-follow-up-cleanups-for-key-failover
> [!NOTE]
> Initially generated by Claude Opus 4.7, modified and reviewed by @ssncferreira
## Description
Separates the aibridge provider configuration from the per-request configuration an interceptor actually needs, and introduces a single `Credential` type that each provider resolves per request. Previously a provider handed its full config to the interceptor (including fields the interceptor didn't use) while other request data was passed as loose arguments, and authentication was spread across config fields and arguments.
## Changes
- Add `intercept.Config`: the per-request, provider-agnostic configuration an interceptor needs (`ProviderName`, `BaseURL`, `APIDumpDir`, `SendActorHeaders`).
- Introduce a single `Credential` interface (`BYOK` and `Centralized`) that each provider resolves per request in `resolveCredential`, and have interceptors route on the credential kind.
- Fail fast with `ErrNoCredential` when a request is neither BYOK nor backed by a centralized key pool.
- Remove unused provider config fields (`Key`, `BYOKBearerToken`, `ExtraHeaders`).
Closes: coder/aibridge#266
Closes: https://linear.app/codercom/issue/AIGOV-221/refactor-separate-provider-and-interceptor-configs
> [!NOTE]
> Initially generated by Claude Opus 4.7, modified and reviewed by @ssncferreira
> AI Tools where used in this request.
Registers `coder_aibridged_*` and `coder_aibridgeproxyd_*` metrics under
new prefixes: `coder_ai_gateway_*` and `coder_ai_gateway_proxy_*`.
Old prefix is still exported. Will be removed in later release.
Also updated the `metricsdocgen` static fixture. Added 4
previously-undocumented metrics `key_pool_state`,
`key_pool_state_transitions_total`, `key_pool_exhaustions_total`,
`key_pool_failover_attempts` added the `client` label to the existing
interception, prompt, and token counter samples.
Updated AI Gateway documentation.
Closes [DOCS-256](https://linear.app/coder/issue/DOCS-256). Sibling to
[DOCS-253](https://linear.app/coder/issue/DOCS-253) (#25740).
Updates docs URL references across the non-TypeScript surface of
`coder/coder` to match the current docs site structure. Source-of-truth
for redirects is `coder/coder.com/redirects.json` (parent ticket
[DOCS-209](https://linear.app/coder/issue/DOCS-209)).
## What changed
| Area | Files | URL mapping |
|---|---|---|
| Top-level README | `README.md` | `/docs/workspaces` ->
`/docs/user-guides/workspace-management`, `/docs/templates` ->
`/docs/admin/templates`, `/docs/ides` ->
`/docs/user-guides/workspace-access` |
| Docs source | `docs/admin/security/0001_user_apikeys_invalidation.md`
| `/docs/admin/audit-logs` -> `/docs/admin/security/audit-logs` |
| Docs source | `docs/install/cloud/azure-vm.md` |
`/docs/coder-oss/latest/install` -> `/docs/install` |
| Dogfood | `dogfood/coder/guide.md` | `/docs/ides` ->
`/docs/user-guides/workspace-access` |
| Helm | `helm/coder/values.yaml` | `/docs/admin/workspace-proxies` ->
`/docs/admin/networking/workspace-proxies` |
| Enterprise coderd | `enterprise/coderd/coderd.go` |
`/docs/admin/encryption` -> `/docs/admin/security/database-encryption`
(error message) |
| Release tooling | `scripts/release/main_internal_test.go` |
`/docs/admin/upgrade` -> `/docs/install/upgrade` (test fixture, matches
`generate_release_notes.sh`) |
| AI bridge | `aibridge/client.go` | repinned to current `main` SHA on
renamed `docs/ai-coder/ai-gateway/monitoring.md`, line range `#L47-L57`
|
| Example templates | 12 `examples/templates/*/README.md`,
`examples/parameters/*`,
`examples/parameters-dynamic-options/README.md`,
`examples/workspace-tags/README.md`, `examples/parameters/main.tf`,
`examples/examples.gen.json` (regenerated) | `/docs/workspaces` ->
`/docs/user-guides/workspace-management`, `/docs/templates/parameters`
-> `/docs/admin/templates/extending-templates/parameters`,
`/docs/templates/dev-containers` ->
`/docs/admin/integrations/devcontainers`, `/docs/dotfiles` ->
`/docs/user-guides/workspace-dotfiles`,
`/docs/about/architecture#agents` ->
`/docs/admin/infrastructure/architecture#agents` |
| Live notification templates (DB) | New migration
`000510_fix_dormancy_notification_docs_urls.up.sql` and `.down.sql` plus
the four regenerated SMTP/webhook goldens under
`coderd/notifications/testdata/rendered-templates/` |
`/docs/templates/schedule#dormancy-threshold-enterprise` ->
`/docs/admin/templates/managing-templates/schedule#dormancy-threshold`,
`/docs/templates/schedule#dormancy-auto-deletion-enterprise` ->
`/docs/admin/templates/managing-templates/schedule#dormancy-auto-deletion`
|
The migration uses `REPLACE(body_template, ...)` scoped by template id
and `LIKE '%/docs/templates/schedule%'`, so it works regardless of which
intermediate state (`000232`, `000262`, `000305`, or `000311`) is
currently in the row.
## What did not change
Historical SQL migrations `000232`, `000262`, `000305`, and `000311` are
not modified because migrations are immutable history. The 18 remaining
stale URL references in those files are superseded at runtime by
migration `000510`. This decision matches the pattern used in the A1
sister PR (#25740).
## Verification
- `go test ./coderd/database/migrations/... -count=1` (UP+DOWN)
- `go test ./coderd/notifications/ -run TestNotificationTemplates_Golden
-update -count=1` to regenerate the four `.golden` files
- `go test ./scripts/release/ -run Test_removeMainlineBlurb -count=1`
- `make pre-commit` (gen + fmt + lint + slim build) ran clean as part of
the commit hook
I also fixed a pre-existing emdash on line 35 of
`examples/templates/azure-linux/README.md` that the lint flagged once
the file entered my diff. The line was already in `main`, but `make gen`
rewrites `examples/examples.gen.json` whenever a `README.md` changes, so
the line came back as a `+` in the diff against `origin/main` and the
`lint/emdash` step refused it.
<details>
<summary>Pre-mortem</summary>
| Risk | Mitigation |
|---|---|
| Migration overwrites future template edits | Used `REPLACE` instead of
full body overwrite. `WHERE id IN (...) AND body_template LIKE
'%/docs/templates/schedule%'` further scopes the write |
| Goldens drift from migrated body | Regenerated goldens via `-update`
after the migration was in place, so the goldens reflect the
post-migration state |
| Down migration leaves stale URLs | Down migration reverses the REPLACE
so a rollback restores the prior URLs |
| Fragment loss when redirect strips fragment | Verified the destination
`schedule.md` contains `## Dormancy threshold` and `## Dormancy
auto-deletion` anchors |
| Terraform parse breakage in `examples/parameters/main.tf` | Only
comments changed; Terraform parser is unaffected |
| Test fixtures in `scripts/release` diverging from
`generate_release_notes.sh` | Updated to match the script, which already
emits `/docs/install/upgrade` |
</details>
---
Generated by Coder Agent on behalf of @nickvigilante.
_Disclosure: produced using Opus 4.8_
I've noticed recently that some prompts are not displaying correctly.
<img width="1388" height="509" alt="image"
src="https://github.com/user-attachments/assets/fad3812f-e42d-4398-a16c-a0e7f924455d"
/>
The cause is the `mid-conversation-system-2026-04-07` beta. When
enabled, the client appends a trailing `role: "system"` message after
the user's text (e.g. an injected skills list):
```json
"messages": [
{ "role": "user", "content": [ "hey how are ya" ] },
{ "role": "system", "content": "The following skills are available..." }
]
```
Our prompt detection algo was being subverted since we only check the
last message if role=user.
AI Bridge reserializes OpenAI chat-completions requests before sending
them upstream. For Gemini OpenAI-compatible routes, that OpenAI
typed-parameter round trip drops
`tool_calls[].extra_content.google.thought_signature`, so Google rejects
tool-result continuations with `Function call is missing a
thought_signature`.
This PR:
- patches the AI Bridge upstream serialization boundary for Gemini
OpenAI-compatible chat completions
- shares the Gemini thought-signature patching helpers with chatd's
OpenAI-compatible transport patch to keep behavior consistent
- treats direct Google OpenAI-compatible upstream endpoints as
Gemini-scoped even when the request model is an alias
- adds the Google fallback thought signature to every assistant tool
call in the active turn, including parallel tool calls
- covers the regression that `extra_content` is dropped before the
upstream body is patched
> Mux updated this PR description on behalf of Mike.
---------
Co-authored-by: Susana Cardoso Ferreira <susana@coder.com>
## Description
This PR adds Prometheus metrics for aibridge's API-key failover, giving visibility into key pool health and failover behavior per provider.
The following metrics are introduced:
- **`key_pool_state`** (gauge): number of keys currently in each state (`valid`, `temporary`, `permanent`) per provider, sampled at scrape time.
- **`key_pool_state_transitions_total`** (counter): key state transitions during failover, labeled by `reason` (`rate_limited`, `unauthorized`, `forbidden`).
- **`key_pool_exhaustions_total`** (counter): times a pool ran out of usable keys, labeled by `outcome` (`rate_limited`, `auth_failed`).
- **`key_pool_failover_attempts`** (histogram): keys attempted before success or exhaustion (per interception for bridged requests, per request for passthrough).
## Changes
- Moves `MarkKeyOnStatus` and key-pool error handling onto `*keypool.Pool`.
- Attaches metrics to each provider's key pool at install time, on construction and on provider reload.
- Adds a scrape-time state collector and a `KeyPools()` accessor on the bridge pool to feed it.
- Tracks per-request key attempts in the bridged and passthrough failover paths.
- Adds test coverage for the new metrics across the keypool unit tests, the bridged intercept failover tests, and the passthrough failover test.
Closes https://github.com/coder/internal/issues/1447
Closes https://linear.app/codercom/issue/AIGOV-198/aibridge-key-failover-observability
> [!NOTE]
> Initially generated by Claude Opus 4.7, modified and reviewed by @ssncferreira
Consolidates the per-interceptor key-failover tests into a single table-driven `keyfailover_test.go`, parameterized over the interceptors (`messages`, `chatcompletions`, `responses`) and modes (blocking and streaming).
It keeps the two scenarios as separate tests: `TestInterception_KeyFailover` (failover within a single interception) and `TestInterception_AgenticLoopFailover` (failover across an agentic-loop continuation). Same cases and assertions as before with far less duplication, reusing the shared `testutil` mocks (`MockUpstream`, `MockServerProxier`, fixture helpers).
Closes https://linear.app/codercom/issue/AIGOV-396/consolidate-intercept-key-failover-tests-across-modes-and-providers
Closes https://linear.app/codercom/issue/AIGOV-395/share-a-single-mockupstream-helper-between-integration-tests-and
> [!NOTE]
> Initially generated by Claude Opus 4.7, modified and reviewed by @ssncferreira
Moves the shared aibridge mock test helpers (`MockUpstream`, `MockServerProxier`, `StubToolCaller`, and the `NewFixtureResponse`/`NewFixtureToolResponse` constructors) out of `aibridge/internal/integrationtest` into `aibridge/internal/testutil`, and exports them.
This lets the per-interceptor test packages (`messages`, `chatcompletions`, `responses`) reuse one set of mocks instead of each redefining its own. The symbols are exported and call sites updated, with no behavior change.
Closes: https://linear.app/codercom/issue/AIGOV-397/move-mockserverproxier-into-a-shared-testutil-package
> [!NOTE]
> Initially generated by Claude Opus 4.7, modified and reviewed by @ssncferreira
The agentic loop has a race between the main goroutine and the `Start`
goroutine on the shared `ResponseWriter`. When an iteration's response
contains only injected-tool events (no text to relay), `Start` may not
have called `InitiateStream` by the time main reaches the `IsStreaming`
check on the next iteration. The `IsStreaming` check then returns false,
main writes a JSON error via `writeUpstreamError`, and `Start` later
writes SSE headers and events on top, producing a malformed JSON+SSE
response:
```
{\"error\":{\"message\":\"all configured keys are rate-limited\",\"type\":\"rate_limit_error\"},\"request_id\":\"\",\"type\":\"error\"}event: message_start\n..."
```
Fix: explicitly call `events.InitiateStream(w)` at the agentic
continuation point so the SSE stream is committed before the next
iteration runs. Keeps `messages` consistent with the pattern already
used in `chatcompletions/streaming.go`. `sync.Once` makes the double
call safe.
Related: coder/internal#1524
Related: coder/coder#25654
Closes:
https://linear.app/codercom/issue/AIGOV-336/flake-teststreaminginterception-agenticloopfailoveragentic-all-keys
> [!NOTE]
> Initially generated by Claude Opus 4.7, modified and reviewed by
@ssncferreira
## Summary
`X-OpenCode-Session` is only set by the OpenCode "Zen" provider. Other
providers use `x-session-affinity` instead. This change falls back to
`x-session-affinity` when `X-OpenCode-Session` is not present, so
sessions are correctly identified regardless of the provider used.
Ref: https://github.com/coder/coder/pull/26128
## Changes
- `aibridge/session.go`: Prefer `X-OpenCode-Session` (Zen), fall back to
`x-session-affinity` (other providers).
- `aibridge/session_test.go`: Add tests for precedence and fallback
behavior.
> Generated with [Coder Agents](https://coder.com) by @dannykopping
This follows #26098 by teaching AI Bridge to read OpenCode session IDs
from the X-OpenCode-Session header, so OpenCode requests are grouped
consistently in interception logs.
Adds unit and integration test coverage for the new header.
<details>
<summary>Coder Agents generated</summary>
This pull request was generated with Coder Agents assistance.
</details>
Adds OpenCode to AI Bridge client detection so requests with user agents
like `opencode/1.16.0 ai-sdk/provider-utils/4.0.23 runtime/bun/1.3.14`
show up as a first-class client instead of `Unknown`.
This also wires the existing OpenCode frontend asset into the AIBridge
UI, adds a Storybook story for the client icon, and updates the
monitoring docs list of supported client values.
<details>
<summary>Coder Agents generated</summary>
This pull request was generated by Coder Agents.
</details>
## Problem
On bridge routes, aibridge acts as a client and originates new outbound
requests via the SDK. Proxy headers (`X-Forwarded-For`,
`X-Forwarded-Host`, etc.) from the inbound client request were forwarded
on the outbound request. The SigV4 signer signs all headers present, so
any in-transit modification by an egress proxy (e.g. appending an IP to
`X-Forwarded-For`) invalidated the signature, causing AWS Bedrock to
reject the request with:
> 403: "The request signature we calculated does not match the signature
you provided."
## Changes
- Strip proxy headers in `PrepareClientHeaders` on bridge routes
- Add unit test for proxy header stripping in `client_headers_test.go`
- Add integration test that verifies SigV4 signature remains valid after
an egress proxy modifies headers in transit
- Add integration test that verifies passthrough routes still set
forwarded headers correctly
Related to internal [Slack
thread](https://codercom.slack.com/archives/C096PFVBZKN/p1779919049215969).
> 🤖 Generated by Coder Agents, modified and reviewed by @ssncferreira
Use testing.Testing() inside createTransport to automatically
clone http.DefaultTransport when running in tests. In production,
DefaultTransport is used as-is (efficient connection pooling).
This fixes the CloseIdleConnections flake class: httptest.Server.Close()
calls http.DefaultTransport.CloseIdleConnections(), which disrupts
any MCP client sharing that transport. The testing.Testing() check
means every MCP transport created during tests gets isolation
automatically, with no caller changes needed.
Closescoder/internal#1016
Closes PLAT-291
## Problem
Centralized requests recorded *the first available key from the pool at
`CreateInterceptor` time* as `credential_hint`, so the interception
could be persisted in the database with a hint that didn't match the key
that actually served the request. The fix consists in storing, at
end-of-interception, the hint of the key that succeeded, or the last
attempted key if all keys are unavailable.
## Changes
- Add `Key.Hint()` and update `credential_hint` on every failover
attempt so it reflects the actually-used key.
- Stop pre-populating `credential_hint` at `CreateInterceptor`.
Centralized starts empty and is updated by the key failover loop.
- Persist the final hint via `RecordInterceptionEnded`; SQL updates
`credential_hint` only when `credential_kind = 'centralized'` so BYOK
keeps its start-time value.
- Log the actually-used hint on interception end/failure; start log uses
a `<keypool-pending>` placeholder for centralized.
> [!NOTE]
> Initially generated by Claude Opus 4.7, modified and reviewed by
@ssncferreira
_Disclosure: created with Coder Agents._
When providers are disabled, we should serve a sentinel error so the
requesting client (Claude Code, Coder Agents, etc) is informed. Coder
Agents can also conditionalize its display to show a helpful error
message.
---------
Signed-off-by: Danny Kopping <danny@coder.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bobby Ho
