We currently have two possible sources of truth for the Bedrock region:
* `cfg.Region`, when explicitly provided (this also covers the case
where the UI parses the base URL and populates `cfg.Region`)
* the region resolved from the AWS environment
An explicitly configured region should always take precedence over the
environment-derived region.
I suggest implementing this resolution policy in the `NewAnthropic`
constructor so that, after initialization, there is a single source of
truth for the resolved region.
Migrate `wg.Add(1); go func() { defer wg.Done(); ... }()` to
`wg.Go(func() { ... })` in tests.
Where the prior pattern passed the loop variable explicitly via a
closure parameter (`go func(id int) { ... }(i)`), drop the parameter and
reference the loop variable directly. Per-iteration loop variables since
Go 1.22 make this safe.
## Description
Updates comments, error strings, and documentation within the
`aibridge/` package to use the new AI Gateway naming, following the
backend rename in #26475.
## Changes
- Update `aibridge/provider/provider.go` comment examples from
`/aibridge` to `/ai-gateway` and "AI Bridge" to "AI Gateway"
- Update `aibridge/AGENTS.md` architecture description
- Update `aibridge/README.md` mount path examples from
`/api/v2/aibridge/` to `/api/v2/ai-gateway/`
- Rename "AI Bridge" to "AI Gateway" in comments across `bridge.go`,
`intercept/client_headers.go`, `intercept/responses/base.go`,
`intercept/messages/base.go`, and `intercept/messages/reqpayload.go`
- Update error string in `intercept/responses/base.go` and matching test
assertion
Addresses
https://github.com/coder/coder/pull/26475#pullrequestreview-4544441981
Refs https://linear.app/codercom/issue/AIGOV-226
> Generated with the assistance of Coder Agents (@ssncferreira)
Bedrock rejects legacy `thinking.type=enabled` requests for Claude Opus
4.8 because the model requires adaptive thinking. The AI Bridge Bedrock
shim only recognized Opus 4.7 as adaptive-only, so Opus 4.8 requests
could fall through and produce Bedrock 400 responses.
Add Opus 4.8 to the adaptive-only model detection and cover the regional
Bedrock model ID form with a regression test.
<details>
<summary>Coder Agents disclosure</summary>
This PR was generated by Coder Agents on behalf of @ericpaulsen.
</details>
# Support IAM role assumption for AWS Bedrock in AI Bridge
## Summary
Implements
https://linear.app/codercom/issue/AIGOV-371/support-dynamic-bedrock-assumerole-across-aws-accounts-for-ai-gateway
A Bedrock provider can now be configured with an IAM role to assume.
Before calling Bedrock, the gateway assumes that role via STS and signs
requests with the resulting temporary credentials. Whether the role
lives in the same account or another one is entirely a matter of the
role's trust policy.
## Problem
Many organizations prohibit long-lived AWS access keys and expect
workloads to authenticate through assumed IAM roles instead. A common
case is an organization that runs Bedrock across several AWS accounts,
one per business unit, and needs each unit's usage billed to its own
account by assuming a role there. AI Bridge previously authenticated a
Bedrock provider only with static keys or the gateway's own ambient AWS
identity, which is shared by every provider, with no way to assume a
role. These deployments had no clean path.
## How it works
When a provider is configured with a role ARN, the gateway uses its base
identity to assume that role via STS and signs Bedrock requests with the
temporary credentials it returns. The base identity is whatever the AWS
default credential chain resolves, IRSA, EKS Pod Identity, EC2 Instance
Profile, or static keys.
Credentials are resolved once when the provider is set up and are then
cached and rotated, so individual requests are served from the cache
rather than triggering a new STS call. A deployment that needs several
roles configures several providers, each pointing at its own role.
## Configuration
The role ARN is part of the Bedrock provider settings and is set through
the AI provider API. It is optional: a provider with no role ARN behaves
exactly as before.
## Scope and trade-offs
- This PR is backend only. The settings UI for the role ARN ships in a
follow-up.
- Configuration is not exposed through environment variables.
Environment-based provider configuration is being phased out in favor of
database-managed providers, so the role ARN is intentionally database
and API only.
Follow-up PR: https://github.com/coder/coder/pull/26578
In our codebase we have an existing convention of using
`flag.Lookup("test.v")` instead of `testing.Testing()`. This avoids
pulling in the entire `testing` package. Another consequence: some of
our custom linters trigger upon import of the `testing` package which
can lead to unexpected linter errors.
PRs https://github.com/coder/coder/pull/26092 and
https://github.com/coder/coder/pull/26519 both landed on main and the
circuit breaker test file ended up importing both
`aibridge/internal/testutil` and `coder/v2/testutil` under the same
name, causing a redeclaration error and breaking `make lint`.
Alias the latter as `codertestutil`, matching the convention already
used in `passthrough_internal_test.go` and `keyfailover_test.go`.
`TestCircuitBreaker_FullRecoveryCycle/OpenAI` flaked once on macOS CI.
The most likely cause is that the circuit breaker `Timeout`
(open-to-half-open transition) was too short relative to the time
between test phases. On a slow runner, the breaker could transition to
half-open before the test verified it was still open, so the request
went through as a half-open probe instead of being rejected.
Increases `Timeout` to `testutil.IntervalMedium` (250ms) across all
circuit breaker integration tests.
**Note:** Ideally, these tests would use a mock clock for deterministic
timing, but https://github.com/sony/gobreaker (the library used for
circuit breaker logic) uses real time internally and doesn't expose a
clock interface.
Closes https://linear.app/codercom/issue/AIGOV-438
> Generated with [Coder Agents](https://coder.com/agents) on behalf of
@ssncferreira
Applies follow-ups from the key pool failover work:
- Add a test verifying key pool state is shared across bridged and passthrough routes.
- Refactor the key failover and passthrough tests to use the shared `MockUpstream` helper.
- Simplify how the request body option is passed through the Anthropic messages interceptor.
- Make `ResponseErrorFromKeyPool` nil-safe and cover it with a test.
Closes: https://linear.app/codercom/issue/AIGOV-398/small-follow-up-cleanups-for-key-failover
> [!NOTE]
> Initially generated by Claude Opus 4.7, modified and reviewed by @ssncferreira
## Description
Separates the aibridge provider configuration from the per-request configuration an interceptor actually needs, and introduces a single `Credential` type that each provider resolves per request. Previously a provider handed its full config to the interceptor (including fields the interceptor didn't use) while other request data was passed as loose arguments, and authentication was spread across config fields and arguments.
## Changes
- Add `intercept.Config`: the per-request, provider-agnostic configuration an interceptor needs (`ProviderName`, `BaseURL`, `APIDumpDir`, `SendActorHeaders`).
- Introduce a single `Credential` interface (`BYOK` and `Centralized`) that each provider resolves per request in `resolveCredential`, and have interceptors route on the credential kind.
- Fail fast with `ErrNoCredential` when a request is neither BYOK nor backed by a centralized key pool.
- Remove unused provider config fields (`Key`, `BYOKBearerToken`, `ExtraHeaders`).
Closes: coder/aibridge#266
Closes: https://linear.app/codercom/issue/AIGOV-221/refactor-separate-provider-and-interceptor-configs
> [!NOTE]
> Initially generated by Claude Opus 4.7, modified and reviewed by @ssncferreira
> AI Tools where used in this request.
Registers `coder_aibridged_*` and `coder_aibridgeproxyd_*` metrics under
new prefixes: `coder_ai_gateway_*` and `coder_ai_gateway_proxy_*`.
Old prefix is still exported. Will be removed in later release.
Also updated the `metricsdocgen` static fixture. Added 4
previously-undocumented metrics `key_pool_state`,
`key_pool_state_transitions_total`, `key_pool_exhaustions_total`,
`key_pool_failover_attempts` added the `client` label to the existing
interception, prompt, and token counter samples.
Updated AI Gateway documentation.
Closes [DOCS-256](https://linear.app/coder/issue/DOCS-256). Sibling to
[DOCS-253](https://linear.app/coder/issue/DOCS-253) (#25740).
Updates docs URL references across the non-TypeScript surface of
`coder/coder` to match the current docs site structure. Source-of-truth
for redirects is `coder/coder.com/redirects.json` (parent ticket
[DOCS-209](https://linear.app/coder/issue/DOCS-209)).
## What changed
| Area | Files | URL mapping |
|---|---|---|
| Top-level README | `README.md` | `/docs/workspaces` ->
`/docs/user-guides/workspace-management`, `/docs/templates` ->
`/docs/admin/templates`, `/docs/ides` ->
`/docs/user-guides/workspace-access` |
| Docs source | `docs/admin/security/0001_user_apikeys_invalidation.md`
| `/docs/admin/audit-logs` -> `/docs/admin/security/audit-logs` |
| Docs source | `docs/install/cloud/azure-vm.md` |
`/docs/coder-oss/latest/install` -> `/docs/install` |
| Dogfood | `dogfood/coder/guide.md` | `/docs/ides` ->
`/docs/user-guides/workspace-access` |
| Helm | `helm/coder/values.yaml` | `/docs/admin/workspace-proxies` ->
`/docs/admin/networking/workspace-proxies` |
| Enterprise coderd | `enterprise/coderd/coderd.go` |
`/docs/admin/encryption` -> `/docs/admin/security/database-encryption`
(error message) |
| Release tooling | `scripts/release/main_internal_test.go` |
`/docs/admin/upgrade` -> `/docs/install/upgrade` (test fixture, matches
`generate_release_notes.sh`) |
| AI bridge | `aibridge/client.go` | repinned to current `main` SHA on
renamed `docs/ai-coder/ai-gateway/monitoring.md`, line range `#L47-L57`
|
| Example templates | 12 `examples/templates/*/README.md`,
`examples/parameters/*`,
`examples/parameters-dynamic-options/README.md`,
`examples/workspace-tags/README.md`, `examples/parameters/main.tf`,
`examples/examples.gen.json` (regenerated) | `/docs/workspaces` ->
`/docs/user-guides/workspace-management`, `/docs/templates/parameters`
-> `/docs/admin/templates/extending-templates/parameters`,
`/docs/templates/dev-containers` ->
`/docs/admin/integrations/devcontainers`, `/docs/dotfiles` ->
`/docs/user-guides/workspace-dotfiles`,
`/docs/about/architecture#agents` ->
`/docs/admin/infrastructure/architecture#agents` |
| Live notification templates (DB) | New migration
`000510_fix_dormancy_notification_docs_urls.up.sql` and `.down.sql` plus
the four regenerated SMTP/webhook goldens under
`coderd/notifications/testdata/rendered-templates/` |
`/docs/templates/schedule#dormancy-threshold-enterprise` ->
`/docs/admin/templates/managing-templates/schedule#dormancy-threshold`,
`/docs/templates/schedule#dormancy-auto-deletion-enterprise` ->
`/docs/admin/templates/managing-templates/schedule#dormancy-auto-deletion`
|
The migration uses `REPLACE(body_template, ...)` scoped by template id
and `LIKE '%/docs/templates/schedule%'`, so it works regardless of which
intermediate state (`000232`, `000262`, `000305`, or `000311`) is
currently in the row.
## What did not change
Historical SQL migrations `000232`, `000262`, `000305`, and `000311` are
not modified because migrations are immutable history. The 18 remaining
stale URL references in those files are superseded at runtime by
migration `000510`. This decision matches the pattern used in the A1
sister PR (#25740).
## Verification
- `go test ./coderd/database/migrations/... -count=1` (UP+DOWN)
- `go test ./coderd/notifications/ -run TestNotificationTemplates_Golden
-update -count=1` to regenerate the four `.golden` files
- `go test ./scripts/release/ -run Test_removeMainlineBlurb -count=1`
- `make pre-commit` (gen + fmt + lint + slim build) ran clean as part of
the commit hook
I also fixed a pre-existing emdash on line 35 of
`examples/templates/azure-linux/README.md` that the lint flagged once
the file entered my diff. The line was already in `main`, but `make gen`
rewrites `examples/examples.gen.json` whenever a `README.md` changes, so
the line came back as a `+` in the diff against `origin/main` and the
`lint/emdash` step refused it.
<details>
<summary>Pre-mortem</summary>
| Risk | Mitigation |
|---|---|
| Migration overwrites future template edits | Used `REPLACE` instead of
full body overwrite. `WHERE id IN (...) AND body_template LIKE
'%/docs/templates/schedule%'` further scopes the write |
| Goldens drift from migrated body | Regenerated goldens via `-update`
after the migration was in place, so the goldens reflect the
post-migration state |
| Down migration leaves stale URLs | Down migration reverses the REPLACE
so a rollback restores the prior URLs |
| Fragment loss when redirect strips fragment | Verified the destination
`schedule.md` contains `## Dormancy threshold` and `## Dormancy
auto-deletion` anchors |
| Terraform parse breakage in `examples/parameters/main.tf` | Only
comments changed; Terraform parser is unaffected |
| Test fixtures in `scripts/release` diverging from
`generate_release_notes.sh` | Updated to match the script, which already
emits `/docs/install/upgrade` |
</details>
---
Generated by Coder Agent on behalf of @nickvigilante.
_Disclosure: produced using Opus 4.8_
I've noticed recently that some prompts are not displaying correctly.
<img width="1388" height="509" alt="image"
src="https://github.com/user-attachments/assets/fad3812f-e42d-4398-a16c-a0e7f924455d"
/>
The cause is the `mid-conversation-system-2026-04-07` beta. When
enabled, the client appends a trailing `role: "system"` message after
the user's text (e.g. an injected skills list):
```json
"messages": [
{ "role": "user", "content": [ "hey how are ya" ] },
{ "role": "system", "content": "The following skills are available..." }
]
```
Our prompt detection algo was being subverted since we only check the
last message if role=user.
AI Bridge reserializes OpenAI chat-completions requests before sending
them upstream. For Gemini OpenAI-compatible routes, that OpenAI
typed-parameter round trip drops
`tool_calls[].extra_content.google.thought_signature`, so Google rejects
tool-result continuations with `Function call is missing a
thought_signature`.
This PR:
- patches the AI Bridge upstream serialization boundary for Gemini
OpenAI-compatible chat completions
- shares the Gemini thought-signature patching helpers with chatd's
OpenAI-compatible transport patch to keep behavior consistent
- treats direct Google OpenAI-compatible upstream endpoints as
Gemini-scoped even when the request model is an alias
- adds the Google fallback thought signature to every assistant tool
call in the active turn, including parallel tool calls
- covers the regression that `extra_content` is dropped before the
upstream body is patched
> Mux updated this PR description on behalf of Mike.
---------
Co-authored-by: Susana Cardoso Ferreira <susana@coder.com>
## Description
This PR adds Prometheus metrics for aibridge's API-key failover, giving visibility into key pool health and failover behavior per provider.
The following metrics are introduced:
- **`key_pool_state`** (gauge): number of keys currently in each state (`valid`, `temporary`, `permanent`) per provider, sampled at scrape time.
- **`key_pool_state_transitions_total`** (counter): key state transitions during failover, labeled by `reason` (`rate_limited`, `unauthorized`, `forbidden`).
- **`key_pool_exhaustions_total`** (counter): times a pool ran out of usable keys, labeled by `outcome` (`rate_limited`, `auth_failed`).
- **`key_pool_failover_attempts`** (histogram): keys attempted before success or exhaustion (per interception for bridged requests, per request for passthrough).
## Changes
- Moves `MarkKeyOnStatus` and key-pool error handling onto `*keypool.Pool`.
- Attaches metrics to each provider's key pool at install time, on construction and on provider reload.
- Adds a scrape-time state collector and a `KeyPools()` accessor on the bridge pool to feed it.
- Tracks per-request key attempts in the bridged and passthrough failover paths.
- Adds test coverage for the new metrics across the keypool unit tests, the bridged intercept failover tests, and the passthrough failover test.
Closes https://github.com/coder/internal/issues/1447
Closes https://linear.app/codercom/issue/AIGOV-198/aibridge-key-failover-observability
> [!NOTE]
> Initially generated by Claude Opus 4.7, modified and reviewed by @ssncferreira
Consolidates the per-interceptor key-failover tests into a single table-driven `keyfailover_test.go`, parameterized over the interceptors (`messages`, `chatcompletions`, `responses`) and modes (blocking and streaming).
It keeps the two scenarios as separate tests: `TestInterception_KeyFailover` (failover within a single interception) and `TestInterception_AgenticLoopFailover` (failover across an agentic-loop continuation). Same cases and assertions as before with far less duplication, reusing the shared `testutil` mocks (`MockUpstream`, `MockServerProxier`, fixture helpers).
Closes https://linear.app/codercom/issue/AIGOV-396/consolidate-intercept-key-failover-tests-across-modes-and-providers
Closes https://linear.app/codercom/issue/AIGOV-395/share-a-single-mockupstream-helper-between-integration-tests-and
> [!NOTE]
> Initially generated by Claude Opus 4.7, modified and reviewed by @ssncferreira
Moves the shared aibridge mock test helpers (`MockUpstream`, `MockServerProxier`, `StubToolCaller`, and the `NewFixtureResponse`/`NewFixtureToolResponse` constructors) out of `aibridge/internal/integrationtest` into `aibridge/internal/testutil`, and exports them.
This lets the per-interceptor test packages (`messages`, `chatcompletions`, `responses`) reuse one set of mocks instead of each redefining its own. The symbols are exported and call sites updated, with no behavior change.
Closes: https://linear.app/codercom/issue/AIGOV-397/move-mockserverproxier-into-a-shared-testutil-package
> [!NOTE]
> Initially generated by Claude Opus 4.7, modified and reviewed by @ssncferreira
The agentic loop has a race between the main goroutine and the `Start`
goroutine on the shared `ResponseWriter`. When an iteration's response
contains only injected-tool events (no text to relay), `Start` may not
have called `InitiateStream` by the time main reaches the `IsStreaming`
check on the next iteration. The `IsStreaming` check then returns false,
main writes a JSON error via `writeUpstreamError`, and `Start` later
writes SSE headers and events on top, producing a malformed JSON+SSE
response:
```
{\"error\":{\"message\":\"all configured keys are rate-limited\",\"type\":\"rate_limit_error\"},\"request_id\":\"\",\"type\":\"error\"}event: message_start\n..."
```
Fix: explicitly call `events.InitiateStream(w)` at the agentic
continuation point so the SSE stream is committed before the next
iteration runs. Keeps `messages` consistent with the pattern already
used in `chatcompletions/streaming.go`. `sync.Once` makes the double
call safe.
Related: coder/internal#1524
Related: coder/coder#25654
Closes:
https://linear.app/codercom/issue/AIGOV-336/flake-teststreaminginterception-agenticloopfailoveragentic-all-keys
> [!NOTE]
> Initially generated by Claude Opus 4.7, modified and reviewed by
@ssncferreira
## Summary
`X-OpenCode-Session` is only set by the OpenCode "Zen" provider. Other
providers use `x-session-affinity` instead. This change falls back to
`x-session-affinity` when `X-OpenCode-Session` is not present, so
sessions are correctly identified regardless of the provider used.
Ref: https://github.com/coder/coder/pull/26128
## Changes
- `aibridge/session.go`: Prefer `X-OpenCode-Session` (Zen), fall back to
`x-session-affinity` (other providers).
- `aibridge/session_test.go`: Add tests for precedence and fallback
behavior.
> Generated with [Coder Agents](https://coder.com) by @dannykopping
This follows #26098 by teaching AI Bridge to read OpenCode session IDs
from the X-OpenCode-Session header, so OpenCode requests are grouped
consistently in interception logs.
Adds unit and integration test coverage for the new header.
<details>
<summary>Coder Agents generated</summary>
This pull request was generated with Coder Agents assistance.
</details>
Adds OpenCode to AI Bridge client detection so requests with user agents
like `opencode/1.16.0 ai-sdk/provider-utils/4.0.23 runtime/bun/1.3.14`
show up as a first-class client instead of `Unknown`.
This also wires the existing OpenCode frontend asset into the AIBridge
UI, adds a Storybook story for the client icon, and updates the
monitoring docs list of supported client values.
<details>
<summary>Coder Agents generated</summary>
This pull request was generated by Coder Agents.
</details>
## Problem
On bridge routes, aibridge acts as a client and originates new outbound
requests via the SDK. Proxy headers (`X-Forwarded-For`,
`X-Forwarded-Host`, etc.) from the inbound client request were forwarded
on the outbound request. The SigV4 signer signs all headers present, so
any in-transit modification by an egress proxy (e.g. appending an IP to
`X-Forwarded-For`) invalidated the signature, causing AWS Bedrock to
reject the request with:
> 403: "The request signature we calculated does not match the signature
you provided."
## Changes
- Strip proxy headers in `PrepareClientHeaders` on bridge routes
- Add unit test for proxy header stripping in `client_headers_test.go`
- Add integration test that verifies SigV4 signature remains valid after
an egress proxy modifies headers in transit
- Add integration test that verifies passthrough routes still set
forwarded headers correctly
Related to internal [Slack
thread](https://codercom.slack.com/archives/C096PFVBZKN/p1779919049215969).
> 🤖 Generated by Coder Agents, modified and reviewed by @ssncferreira
Use testing.Testing() inside createTransport to automatically
clone http.DefaultTransport when running in tests. In production,
DefaultTransport is used as-is (efficient connection pooling).
This fixes the CloseIdleConnections flake class: httptest.Server.Close()
calls http.DefaultTransport.CloseIdleConnections(), which disrupts
any MCP client sharing that transport. The testing.Testing() check
means every MCP transport created during tests gets isolation
automatically, with no caller changes needed.
Closescoder/internal#1016
Closes PLAT-291
## Problem
Centralized requests recorded *the first available key from the pool at
`CreateInterceptor` time* as `credential_hint`, so the interception
could be persisted in the database with a hint that didn't match the key
that actually served the request. The fix consists in storing, at
end-of-interception, the hint of the key that succeeded, or the last
attempted key if all keys are unavailable.
## Changes
- Add `Key.Hint()` and update `credential_hint` on every failover
attempt so it reflects the actually-used key.
- Stop pre-populating `credential_hint` at `CreateInterceptor`.
Centralized starts empty and is updated by the key failover loop.
- Persist the final hint via `RecordInterceptionEnded`; SQL updates
`credential_hint` only when `credential_kind = 'centralized'` so BYOK
keeps its start-time value.
- Log the actually-used hint on interception end/failure; start log uses
a `<keypool-pending>` placeholder for centralized.
> [!NOTE]
> Initially generated by Claude Opus 4.7, modified and reviewed by
@ssncferreira
_Disclosure: created with Coder Agents._
When providers are disabled, we should serve a sentinel error so the
requesting client (Claude Code, Coder Agents, etc) is informed. Coder
Agents can also conditionalize its display to show a helpful error
message.
---------
Signed-off-by: Danny Kopping <danny@coder.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds middleware in `withAWSBedrockOptions` that appends the AWS Partner
Revenue Measurement (PRM) attribution string to the User-Agent header on
every Bedrock API call made through AI Bridge.
This is the AI Bridge counterpart to the Terraform provisioner change
merged in #23138. Together, they ensure all AWS API calls made by Coder
(both workspace infrastructure via Terraform and LLM inference via
Bedrock) include PRM attribution.
## How it works
- A middleware is added before `bedrock.WithConfig(awsCfg)` that reads
the existing `User-Agent` header and appends
`sdk-ua-app-id/APN_1.1%2Fpc_cdfmjwn8i6u8l9fwz8h82e4w3%24`
- Only affects Bedrock calls; OpenAI and direct Anthropic API calls are
unaffected
- Uses `option.WithMiddleware` rather than `option.WithHeader` because
the existing User-Agent (set by the Anthropic SDK) must be preserved and
appended to, not replaced
## Tests
- **Positive**: `TestAWSBedrockIntegration` verifies PRM attribution is
present in the User-Agent on Bedrock requests
- **Negative**: `TestAnthropicMessages` verifies PRM attribution is
absent on non-Bedrock requests
## References
- Companion Terraform provisioner PR: #23138 (merged)
- Backport: #24052 (merged)
- Preserve existing `AWS_SDK_UA_APP_ID`: #24606 (open)
- Original `coder/aibridge` PR:
https://github.com/coder/aibridge/pull/224 (superseded by this PR since
aibridge was moved into coder/coder via #24190)
- [AWS SDK Application ID
docs](https://docs.aws.amazon.com/sdkref/latest/guide/feature-appid.html)
- [AWS PRM Automated User
Agent](https://prm.partner.aws.dev/automated-user-agent.html) (partner
login required)
> Generated with [Coder Agents](https://coder.com/agents)
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
## Description
`Provider.InjectAuthHeader` is no longer needed. With the addition of `KeyFailoverConfig` in #24920, authentication is now applied per-attempt by `KeyFailoverTransport` on passthrough routes. This PR removes the dead method from the `Provider` interface, all implementations (`Anthropic`, `OpenAI`, `Copilot`), and the test mock.
The orphaned `InjectAuthHeader` unit tests are replaced with `Test{Anthropic,OpenAI,Copilot}_KeyFailoverConfig`. `TestPassthrough_KeyFailover` is also extended to cover Copilot in the BYOK scenario.
Related to: https://linear.app/codercom/issue/AIGOV-334/aibridge-follow-ups-from-key-failover-prs
> [!NOTE]
> Initially generated by Claude Opus 4.7, modified and reviewed by @ssncferreira
## Description
Cleans up how key pool errors are represented and how they get turned into HTTP responses. Consolidates two error types into a single type with a kind tag, and gives the response helpers in both providers consistent names.
## Changes
- Replaced the keypool sentinel and transient error struct with one error type that carries a kind and a retry-after duration.
- Updated `KeyFailoverConfig.BuildKeyPoolResponse` to take the typed key pool error, so each provider can shape the exhaustion response in its own format.
- Removed the per-provider `MarkKey` callback from `KeyFailoverConfig` since providers can rely on the shared `MarkKeyOnStatus` helper.
- Renamed the response-error helpers so OpenAI and Anthropic use the same naming.
Related to: https://linear.app/codercom/issue/AIGOV-334/aibridge-follow-ups-from-key-failover-prs
> [!NOTE]
> Initially generated by Claude Opus 4.7, modified and reviewed by @ssncferreira
Documents the known race in `EventStream.IsStreaming()` and the
resulting flake in
`TestStreamingInterception_AgenticLoopFailover/agentic_all_keys_fail `,
accepted rather than fixed since the inner agentic loop is on track to
be removed as part of the reverse proxy migration in coder/aibridge#223.
Full reasoning in coder/internal#1524.
My agent added `//nolint:testpackage` to a test file on one of my PRs.
Again. This PR cleans it up across the entire repo and updates the
in-repo conventions so future agents stop doing it.
The repo already has a precedent for white-box tests that need to touch
unexported symbols: `*_internal_test.go` (145+ existing files). The
`testpackage` linter's default `skip-regexp` exempts that filename
suffix, so the `//nolint:testpackage` directive is unnecessary in every
case where someone reached for it. This PR renames 51 such files to
`*_internal_test.go` via `git mv` so blame and history follow, and
strips the dead directive from 2 files that were already correctly named
(`coderd/oauth2provider/authorize_internal_test.go`,
`coderd/x/chatd/advisor_internal_test.go`).
`.claude/docs/TESTING.md` now documents the rule explicitly under *Test
Package Naming*, which is imported into the root `AGENTS.md` via
`@.claude/docs/TESTING.md`. The rule: prefer `package foo_test`; if you
need internal access, rename the file to `*_internal_test.go` rather
than adding a nolint directive.
*Disclaimer: implemented by a Coder Agent using Claude Opus 4.6/4.7*
Fixes
[coder/aibridge#280](https://github.com/coder/aibridge/issues/280).
Claude Opus 4.7 (and future adaptive-only Bedrock models) reject the
legacy `thinking.type: "enabled"` + `budget_tokens` shape with a 400.
Claude Code falls back to that shape when it cannot read the upstream
model's capability metadata, which is exactly the case when AI Bridge
sits between the client and Bedrock. Pinning back to Opus 4.6 is the
only operator workaround today.
This is the counterpart to the `adaptive -> enabled` conversion added in
[coder/aibridge#225](https://github.com/coder/aibridge/pull/225) for
older Bedrock models.
## Behavior
- New `bedrockModelRequiresAdaptiveThinking()` helper matches Opus 4.7
(covers `us.anthropic.claude-opus-4-7`, ARN-style application inference
profile names that include the model ID, etc.).
- New `RequestPayload.convertEnabledThinkingForBedrock()` rewrites
`thinking: {type: enabled, budget_tokens: N}` to `thinking: {type:
adaptive}`. The budget hint is dropped; an explicit
`output_config.effort` from the caller is preserved naturally because we
never touch that field. We deliberately do **not** derive an effort
label from the budget (see decision log).
- `removeUnsupportedBedrockFields` learns a variadic `exemptFields`
parameter. Adaptive-only models support `output_config` natively (no
beta flag required), so `augmentRequestForBedrock` exempts that field
for those models.
- Bedrock Opus 4.7 accepts `output_config.effort` but rejects
`output_config.format` (structured outputs) with the same "Extra inputs
are not permitted" 400. The generic strip pass operates at top-level
granularity only, so a small targeted pass drops `output_config.format`
after the top-level strip for adaptive-only models.
The whole Bedrock thinking-type shim block carries a header comment
flagging it as temporary; a planned native Bedrock provider removes the
impedance mismatch and lets us delete it.
## Out of scope
The issue calls out a possible follow-up around `Anthropic-Beta:
interleaved-thinking-2025-05-14` for adaptive-only models; best evidence
is that Opus 4.7 still accepts those flags, so this PR is a no-op there.
<details>
<summary>Decision log</summary>
- `bedrockModelSupportsAdaptiveThinking` now also returns `true` for
adaptive-only models. That keeps the existing
`convertAdaptiveThinkingForBedrock` branch from running on Opus 4.7
(which would otherwise be incorrect; `adaptive` is the supported native
type there), and the new `convertEnabledThinkingForBedrock` runs only
for adaptive-only models via the explicit
`bedrockModelRequiresAdaptiveThinking` switch case. The two model sets
are disjoint by construction.
- The reverse conversion does **not** derive `output_config.effort` from
`budget_tokens / max_tokens`. The two thinking shapes encode different
intents (`enabled+budget` is "give me exactly N tokens,"
`adaptive[+effort]` is "model, pick a budget, optionally biased") and
there is no canonical mapping between them. An earlier draft of this PR
derived effort via midpoints of an invented anchor table; it was
symmetric-looking but lossy and required a lot of scaffolding (sorted
anchors, init-time invariant guard, round-trip tests) to keep two halves
consistent. The reverse direction now just rewrites the shape, which is
honest about the information loss and matches platform-defined adaptive
behavior when no effort hint is present.
- `output_config.format` is stripped only for adaptive-only models.
Other Bedrock models either don't get `output_config` through at all
(top-level strip handles them) or accept it via a beta flag that may
imply broader feature support. Easy to widen if the same 400 shows up
elsewhere.
- I chose `variadic exemptFields ...string` over passing the model down
to `removeUnsupportedBedrockFields`, to keep that function focused on
stripping and to localise the model-aware policy in
`augmentRequestForBedrock`.
</details>
## Description
Adds automatic key failover for passthrough routes for the Anthropic and OpenAI providers. A new `keyFailoverTransport` wraps the reverse-proxy transport: centralized requests walk the configured key pool and retry with the next key on key-specific failures (401/403/429), reusing the same key-marking semantics as the bridged routes.
BYOK passthrough requests run as a single attempt with no failover.
## Changes
- New `keypool.KeyFailoverConfig` carrying the `Pool` to walk and the provider-specific closures (`IsBYOK`, `InjectAuthKey`, `MarkKey`, `BuildExhaustedResponse`).
- New `keypool.NewKeyFailoverTransport`: wraps an inner `http.RoundTripper`. Returns `inner` unchanged when `Pool` is nil, otherwise produces a transport that buffers the request body once, walks the pool per request, and replays each attempt with the next key.
- New `Provider.KeyFailoverConfig(logger)` interface method. Anthropic injects `X-Api-Key`; OpenAI injects `Authorization: Bearer ...`; Copilot returns an empty config.
- `passthrough.go` wires `NewKeyFailoverTransport` around the existing apidump middleware, so every retry attempt is recorded.
## Related Issues
Related to: https://github.com/coder/internal/issues/1446
Related to: https://linear.app/codercom/issue/AIGOV-197/aibridge-automatic-key-failover-for-bridged-and-passthrough-routes
## Follow-up PRs
- Remove dead `Provider.InjectAuthHeader` method now that all auth is applied per-attempt by `KeyFailoverTransport`.
- Bedrock multi-key support.
- Refactor provider vs interceptor config separation.
- Record the actually-used key in the interception credential hint after failover.
> [!NOTE]
> Initially generated by Claude Opus 4.7, modified and reviewed by @ssncferreira
## Description
Adds automatic key failover for centralized OpenAI provider, covering both chat completions and responses APIs. Same shape as the Anthropic PR: each upstream call walks the configured key pool, keys are marked **temporary** on 429 (with cooldown from `Retry-After`) and **permanent** on 401/403. Each agentic-loop iteration gets its own fresh walker so a tool-call continuation can fail over independently of the initial request.
BYOK is unchanged: BYOK requests run as a single attempt with no failover.
## Changes
- `config.OpenAI` carries a `KeyPool`. `Key` remains for BYOK Authorization Bearer set per interception.
- Chat completions blocking interceptor: walks the pool via `newChatCompletionWithKeyFailover`, marks keys on key-specific failures, returns on first success or non-failover error.
- Chat completions streaming interceptor: per-iteration walker. Pre-stream failures fail over to the next key; mid-stream errors are relayed as SSE events.
- Responses blocking interceptor: extracts `newResponseWithKeyFailover` parallel to chatcompletions.
- Responses streaming interceptor: per-iteration walker, retains the existing buffer-then-forward design.
## Related Issues
Related to: https://github.com/coder/internal/issues/1446
Related to: https://linear.app/codercom/issue/AIGOV-197/aibridge-automatic-key-failover-for-bridged-and-passthrough-routes
## Follow-up PRs
- Bedrock multi-key support.
- Refactor provider vs interceptor config separation.
- Record the actually-used key in the interception credential hint after failover.
> [!NOTE]
> Initially generated by Claude Opus 4.7, modified and reviewed by @ssncferreira
## Description
Adds automatic key failover for centralized Anthropic provider. When a key pool is configured, each upstream call walks the pool and tries keys in order until one succeeds or the pool is exhausted. Keys are marked **temporary** on 429 (with cooldown from `Retry-After`) and **permanent** on 401/403. Errors that aren't key-specific don't trigger failover. Each agentic-loop iteration gets its own fresh walker, so a tool-call continuation can fail over independently of the initial request.
BYOK is unchanged: BYOK requests run as a single attempt with no failover.
## Changes
- `config.Anthropic` carries a `KeyPool`. `Key` remains for BYOK X-Api-Key set per interception.
- Blocking interceptor: walks the pool, marks keys on key-specific failures, returns on first success or non-failover error.
- Streaming interceptor: per-iteration walker. Pre-stream failures fail over to the next key; mid-stream errors are relayed as SSE events.
- New `keypool` error types: `TransientExhaustionError` (carries soonest cooldown) and `ErrPermanentExhaustion`. Replace the prior `ErrAllKeysExhausted`.
- Error responses now consistently include the outer `"type": "error"` field.
## Related Issues
Related to: https://github.com/coder/internal/issues/1446
Related to: https://linear.app/codercom/issue/AIGOV-197/aibridge-automatic-key-failover-for-bridged-and-passthrough-routes
## Follow-up PRs
- Bedrock multi-key support.
- Refactor provider vs interceptor config separation.
- Record the actually-used key in the interception credential hint after failover.
> [!NOTE]
> Initially generated by Claude Opus 4.7, modified and reviewed by @ssncferreira
## Description
Removes 429 (Too Many Requests) from the circuit breaker failure conditions. Rate limiting is now handled by automatic key failover instead of tripping the circuit breaker.
## Changes
`DefaultIsFailure` no longer treats 429 as a circuit breaker failure. The circuit breaker now only trips on server overload responses (503, 529).
Tests and integration tests updated to use 503 instead of 429 for tripping circuits. Description strings in deployment config updated to reflect the change.
Closes https://github.com/coder/internal/issues/1445
> [!NOTE]
> Initially generated by Coder Agents, modified and reviewed by @ssncferreira
## Description
Adds the `aibridge/keypool` package, a thread-safe key pool with per-key state tracking and cooldown expiry. This PR introduces the package only; wiring it into the aibridge providers and coder configuration will happen in upstream PRs.
## Changes
Each key is in one of three states: **Valid** (available), **Temporary** (rate-limited with cooldown expiry), or **Permanent** (revoked/unauthorized, terminal until restart). The state is derived from the key fields rather than stored explicitly: once a cooldown expires, the key is valid again without any external action.
A `Walker` provides per-request key traversal using a primary-with-fallback strategy: each request walks the pool from index 0, skipping unavailable keys, so the first key is always preferred when healthy. Walkers are independent, so concurrent requests traverse the pool without interfering with each other.
`MarkTemporary` preserves the longer cooldown when concurrent requests both mark the same key.
Relates to https://github.com/coder/internal/issues/1445
> [!NOTE]
> Initially generated by Coder Agents, modified and reviewed by @ssncferreira
> AI tools where used when creating this PR
This PR removes environment variable parsing from `/aibridge` directory.
Added env variables/flags for dump dir as coder options.
Only added to new indexed provider options
(`CODER_AIBRIDGE_PROVIDER_<N>_*`) not to deprecated legacy env variables
(`CODER_AIBRIDGE_ANTHROPIC_*` and `CODER_AIBRIDGE_OPENAI_KEY_*`).
Reverted adding `MaxRetries` option as it will be removed soon due to
key failover work:
https://github.com/coder/coder/pull/24783#discussion_r3155544808
> AI tools where used when creating this PR
This PR:
* removes references to aibridge repository from coder docs
* updates aibdrige/README.md
* makes it clear aibridge (keeping old name) is a handler not a separate
process
* updates outdated sections about: metrics, recorded interface and
supported paths.
---------
Co-authored-by: Susana Ferreira <susana@coder.com>
> AI tools where used when creating this PR
Updates `AGENTS.md` to reflect that `/aibridge` is now a directory in
coder/coder repo not a separate repo.
Yevhenii Shcherbina
