Categorises the terminal error of a failed interception and persists it
on the interception record, then surfaces it on the AI Gateway API.
- Categorise into an enum (`bad_request`, `unauthorized`,
`rate_limited`, `overloaded`, `server_error`, `unknown`), unwrapping
the ResponseError envelope, the upstream Anthropic/OpenAI SDK errors,
and key-pool exhaustion so blocking and streaming paths agree.
- Thread the type and raw message through the recorder dRPC into the
`aibridge_interceptions` row (optional proto fields; NULL on success).
- Expose the error on the AI Gateway thread API from the root
interception.
*This PR was produced by opencode (agent) using the `anthropic/claude-opus-4-8` model, under human direction and review.*
## Why
PR #26797 was accidentally merged into the stale `graphite-base/26797`
branch instead of `main` (Graphite picked the wrong base), so its
changes never landed on `main`. This PR re-lands that work as a clean
cherry-pick onto the current `main`.
## What
Adds a `WatchAIProviders` streaming RPC to the `ProviderConfigurator`
service so a running standalone AI Gateway refetches its provider set
when the provider configuration changes. The server subscribes to
`AIProvidersChangedChannel` (published by the provider CRUD endpoints)
and forwards each event as a payload-free signal, plus one signal on
subscribe; the gateway calls `GetAIProviders` on each signal to rebuild
its pool. The aibridged API is bumped to v1.2.
Env-seeded providers don't need a signal: seeding finishes before coderd
serves the gateway connection, so the gateway's initial fetch already
reflects the seeded set.
## For reviewers
The change is split into two commits to make review easy:
1. **`feat: synchronise provider changes with WatchAIProviders`** is a
faithful cherry-pick of #26797, identical to the originally reviewed PR.
It is committed without pre-commit hooks because it does not build
against current `main` on its own.
2. **`fix: resolve cherry-pick conflicts against main`** contains only
the deltas needed to re-land on current `main`, and passes the full
pre-commit suite:
- `coderd/aibridged/proto/aibridged.pb.go` regenerated via the proto
make target (the cherry-picked copy was generated against the older
proto).
- `enterprise/cli/aigatewaystart.go` import block unioned; `main` added
`os` and `strings` while the PR added `sync`.
- Three `aibridgedserver.NewServer` test call sites that landed on
`main` after the original branch diverged now pass the new `pubsub`
argument.
Refs https://linear.app/codercom/issue/AIGOV-465
*This PR was produced by opencode (agent) using the
`anthropic/claude-opus-4-8` model, under human direction and review.*
## Description
Extracts the AI budget period computation into a shared
`budget.CurrentPeriod` helper. Pure refactor with no wire-value changes.
## Changes
- Add `budget.CurrentPeriod(now, period)` returning a
`PeriodWindow{Start, End}` in UTC. Unknown periods return an error,
matching the pattern used by `ResolveUserAIBudget` for unknown policies.
- Update the callers and respective tests to use `CurrentPeriod`.
> [!NOTE]
> Initially generated by Claude Opus 4.7, modified and reviewed by
@ssncferreira
## Summary
Plumbs the Responses output item id (added as `ToolUsageRecord.ItemID` in #26855) through to the database, captured independently of the `provider_tool_call_id` correlation key. Hosted tools (`web_search_call`, etc.) only have an item id; agentic tools have both.
`provider_item_id` is specific to the OpenAI Responses API; it stays empty for chat completions and Anthropic messages, which have no separate item id.
## Changes
- Migration `000534`: nullable `provider_item_id` column on `aibridge_tool_usages`.
- Proto: `item_id` field 11 on `RecordToolUsageRequest`.
- Server handler: persists `provider_item_id` and adds it to structured logging.
- Translator: maps `ToolUsageRecord.ItemID` to the proto field.
## Tests
- `TestRecordToolUsageProviderItemID`: real-database round-trip asserting `provider_item_id` persists for both hosted and agentic tools, independently of `provider_tool_call_id`.
Stacked on #26855. Linear: AIGOV-96
---
_This PR was produced by opencode (agent) using the_ _`anthropic/claude-opus-4-8`_ _model, under human direction and review._
## Description
Adds pre-request AI budget enforcement to `aibridged`. Requests are rejected with HTTP 403 when the user's aggregated spend for the current period has reached their effective limit.
## Changes
- Add `IsBudgetExceeded` RPC to `aibridgedserver`. Resolves the user's effective budget, aggregates spend over the caller-supplied `[period_start, now]` window, and returns whether the limit has been reached along with the effective limit.
- Wire the check into `aibridged`'s HTTP handler. The caller computes the period start (monthly for now) and passes it in the request.
- Reject exceeded requests with HTTP 403 Forbidden and a message directing the user to contact an administrator.
- Add `dbtime.StartOfMonth` alongside `StartOfDay` for period computation.
- Add real-DB tests covering the enforcement path: month-boundary excludes prior-period spend, and a new user override unblocks a previously-exceeded user.
Closes https://linear.app/codercom/issue/AIGOV-428/add-pre-request-budget-enforcement
> [!NOTE]
> Initially generated by Claude Opus 4.7, modified and reviewed by @ssncferreira
## Description
Adds post-response spend accumulation to `RecordTokenUsage`.
## Changes
- Wrap the token usage insert and daily spend increment in a single transaction.
- Skip the spend update when the user is unbudgeted, the model is unpriced, or the computed cost is non-positive.
Depends on #26562
Closes https://linear.app/codercom/issue/AIGOV-427/add-post-response-spend-accumulation
> [!NOTE]
> Initially generated by Claude Opus 4.7, modified and reviewed by @ssncferreira
Implements:
https://linear.app/codercom/issue/AIGOV-495/add-externalid-to-prevent-confused-deputy-problem
When a Bedrock provider assumes an IAM role via STS, the gateway now
generates a unique external ID for it and sends that value on every
`AssumeRole` call. The external ID guards against the [confused deputy
problem](https://docs.aws.amazon.com/IAM/latest/UserGuide/confused-deputy.html)
on cross-account role assumption. Per [AWS's
recommendation](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-user_externalid.html),
the gateway generates and owns the value rather than accepting one from
the operator; that ownership is what makes it effective, since a party
who knows another's external ID can't induce the gateway to send it.
The external ID is server-owned and read-only over the API. It is
generated once, when a provider first has a `role_arn`, and is stable
thereafter. Clients cannot set it: create rejects any supplied
`external_id`, and update rejects a value that differs from the stored
one. An update may echo the stored value back unchanged, so the normal
read-modify-write flow (GET the provider, change a field, PATCH the full
settings object) keeps working. The value is not a secret and is
returned on GET so operators can copy it into the target role's trust
policy as an `sts:ExternalId` condition.
It is persisted in the existing JSON settings blob, so there is no
migration or audit-table change.
Closes [AIGOV-455](https://linear.app/codercom/issue/AIGOV-455/extend-drpc-with-buildproviders).
## Why
The AI Gateway (`aibridged`) is being split into a standalone process that must not touch the database. `coderd` stays the source of truth and seeds the `ai_providers` / `ai_provider_keys` tables from the environment. This PR adds a DRPC call so the gateway fetches provider config from `coderd` instead of reading the DB, for both the embedded and standalone daemons.
## What
- **Proto:** new `ProviderConfigurator` service with a unary `GetAIProviders` RPC, plus `AIProvider` / `AIProviderBedrock` messages. `CurrentMinor` bumped to 1 (additive).
- **Server (`coderd/aibridgedserver`):** `GetAIProviders` runs a read-only `InTx` under `LockIDAIProvidersEnvSeed` so it never returns a mid-seed snapshot, reads providers (incl. disabled) plus keys for enabled ones, and maps to proto under `dbauthz.AsAIBridged`. Unmappable rows are skipped and logged; plaintext keys and Bedrock secrets are never logged.
- **Client:** `DRPCProviderConfiguratorClient` wired into the client union, `dialer.go`, and `CreateInMemoryAIBridgeServer`.
- **cli:** `BuildProvidersFromProto` maps the response through the existing DB-neutral `buildProvider`. A shared `poolRPCReloader` does the fetch/build/replace for both daemons: the embedded daemon reloads on every `ai_providers` change and fails startup if it cannot subscribe; the standalone gateway drives the same reloader once at startup, retrying until success and staying interruptible.
- **Dead code removed:** `BuildProvidersFromConfig`, `ProvidersFromConfig`, `AIProviderFromConfig`, and the DB-read `BuildProviders` path.
Adds a new enterprise-only `GET /api/v2/ai-gateway/serve` endpoint that standalone AI Gateway replicas use to connect to `coderd` over a DRPC-over-WebSocket transport, mirroring the existing in-memory path used by the embedded AI Bridge daemon.
- The endpoint upgrades the HTTP connection to a WebSocket, multiplexes it with yamux, and finally serves the three DRPC services (Recorder, MCPConfigurator, Authorizer).
- The `X-AI-Governance-Gateway-Key` header is used for authentication.
- The key is looked up by its hashed secret
- Missing or revoked keys return `401`.
- API version negotiation is enforced via a new `aibridged/proto` version (`v1.0`).
- Incompatible versions return `400`.
- `FeatureAIBridge` entitlement is required.
- Key liveness (`last_used_at`) is recorded immediately on connection and refreshed every 60 seconds while the session remains open.
- When key liveness detects the key was deleted (no rows where updated) session is closed.
#### Small refactors
* The three DRPC service registrations are extracted into `aibridgedserver.Register`, shared by both the in-memory and WebSocket paths.
* The literal `256 * 1024` used as the yamux-aligned WebSocket read limit is replaced with the named constant `drpcsdk.YamuxDefaultStreamWindowSize` in all call sites.
* as noted in review comment https://github.com/coder/coder/pull/26506#discussion_r3461905223 order of `SetReadLimit` and `WebsocketNetConn` calls was fixed.
Implements
https://linear.app/codercom/issue/AIGOV-286/add-interception-cost-calculation-to-aibridge-token-usages
Adds spend attribution to AI Gateway. After the upstream response, each
token-usage record now captures the user's effective group, the
per-token prices in effect at that moment, and a computed cost — so
spend is recorded as an immutable, point-in-time snapshot.
Concretely, `aibridge_token_usages` gains `effective_group_id`,
`input_price_micros`, `output_price_micros`, `cache_read_price_micros`,
`cache_write_price_micros`, and `cost_micros`. When a usage record is
written, the effective group is resolved (per-user override, else the
deployment budget policy), the `(provider, model)` price is looked up
and snapshotted onto the row, and cost is computed from the
provider-reported token counts. A model that isn't in the price table
records its tokens with a `NULL` cost; any *other* resolution failure
fails the write, so a `NULL` cost unambiguously means "model not priced"
rather than "lookup errored."
All values are stored in micro-units (1 unit = 1,000,000 micro-units;
Phase 1 assumes USD, so 1 micro-unit = $0.000001). Prices are quoted per
million tokens.
This also grants the AI Bridge RBAC subject `read` on `ai_model_prices`
(the per-interception price lookup needs it; it previously only had
`update` for the startup seeder).
## Cost precision
Cost is computed per token category as `tokens × price / 1_000_000` with
integer division, then the four categories are summed. The division is
done **per category** (not once over the summed numerator) on purpose:
it keeps the per-category line items summing exactly to the stored total
— no "the parts don't add up to the whole" in reporting).
Integer division truncates sub-micro-unit fractions. For example, a
cheap model at $0.10 per million tokens is a price of `100_000`; 9
tokens cost `9 × 100_000 / 1_000_000 = 900_000 / 1_000_000 = 0` (the
true 0.9 micro-units floors to 0). At real list prices this rarely bites
— $3/M input is a price of `3_000_000`, so even a single token is 3
micro-units. The per-record under-count is bounded below 1 micro-unit
per category, so under $0.000004 total across the four categories, which
is acceptable for list-price-based cost approximation.
## Overflow safety
`cost_micros` is a `BIGINT` (int64), and the largest intermediate value
is a single category's `tokens × price` before division. int64's ceiling
is ≈ `9.223e18`.
- At a steep $75/M model (price `75_000_000`), overflow would require
~123 billion tokens in one response: `123e9 × 75e6 = 9.225e18`, just
over the limit. `122e9` stays under at `9.15e18`.
- A realistically maxed-out Opus 4.8 response (≈1M input + 128K output
at list prices) costs about $15, with a numerator around `1.5e13` —
roughly six orders of magnitude below the ceiling.
So overflow is unreachable from real token counts.
### Multi-currency support
In the future, we may encounter issues with multi-currency support,
especially when dealing with currencies that have very large exchange
rates relative to USD, for example:
IRR: ~1,300,000 IRR ≈ 1 USD
VND: ~26,000 VND ≈ 1 USD
For currencies with such large denominations, numeric overflow is
technically possible, considering that we have only about six orders of
magnitude of headroom before reaching the limit (see above).
## `effective_group_id` has no foreign key
`effective_group_id` records the group a spend was attributed to, as an
immutable historical fact. It is intentionally **not** a foreign key, so
the record survives deletion of the group.
Alternatives were considered and rejected:
- **`ON DELETE SET NULL`** would mutate an "immutable" record — deleting
a group silently erases that interception's attribution and under-counts
the group's historical spend.
- **`RESTRICT` / `NO ACTION`** would block group deletion entirely
(groups are hard-deleted).
- **`CASCADE`** would delete spend history when a group is deleted — the
worst outcome for an audit record.
There is also no insert-time check that the group still exists: the id
comes from a budget that was just resolved, meaning it was valid at some
point.
## Open question: group name snapshotting
Should we also snapshot the group *name* onto each record? Two options:
- **Denormalize it now** — readable in historical reports even after a
group is deleted, but the snapshot can drift from the current name on
rename, raising a "show point-in-time vs. current name" question.
- **Postpone until needed** — it's a purely additive column later, and
the name is display-only (not correctness-bearing like the price). The
cost: names of groups deleted before the column is added can't be
backfilled.
Leaning toward postponing until a concrete reporting need settles the
drift question.
Add `agent_firewall_session_id` (UUID NULL) and
`agent_firewall_sequence_number` (INT NULL) to `aibridge_interceptions`
with a partial index on `agent_firewall_session_id`. No FK to
`boundary_sessions` (soft reference, resolved at query time).
`RecordInterception` reads the new fields from the proto request (merged
in #25884) via `parseOptionalUUID` / `parseOptionalInt32` helpers.
> This PR was authored by Coder Agents.
## Problem
Centralized requests recorded *the first available key from the pool at
`CreateInterceptor` time* as `credential_hint`, so the interception
could be persisted in the database with a hint that didn't match the key
that actually served the request. The fix consists in storing, at
end-of-interception, the hint of the key that succeeded, or the last
attempted key if all keys are unavailable.
## Changes
- Add `Key.Hint()` and update `credential_hint` on every failover
attempt so it reflects the actually-used key.
- Stop pre-populating `credential_hint` at `CreateInterceptor`.
Centralized starts empty and is updated by the key failover loop.
- Persist the final hint via `RecordInterceptionEnded`; SQL updates
`credential_hint` only when `credential_kind = 'centralized'` so BYOK
keeps its start-time value.
- Log the actually-used hint on interception end/failure; start log uses
a `<keypool-pending>` placeholder for centralized.
> [!NOTE]
> Initially generated by Claude Opus 4.7, modified and reviewed by
@ssncferreira
Allows an `api_key_id` to be passed from a trusted in-memory transport
(currently: `chatd`) to `aibridged` for use in authenticating LLM
requests.
This value can _only_ be passed via context, and all users of the
in-memory transport _must_ provide it.
It can be used in conjunction with BYOK headers.
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
In order to allow Coder Agents to use AI Gateway in OSS, we need to rehome the `aibridged`\-related code into the AGPL path.
The HTTP API is only registered under enterprise so will still require the AI Governance Add-on to be present in order to use it, whereas Coder Agents uses an in-memory pipe to the same handlers.
Danny Kopping