## Why
PR #26797 was accidentally merged into the stale `graphite-base/26797`
branch instead of `main` (Graphite picked the wrong base), so its
changes never landed on `main`. This PR re-lands that work as a clean
cherry-pick onto the current `main`.
## What
Adds a `WatchAIProviders` streaming RPC to the `ProviderConfigurator`
service so a running standalone AI Gateway refetches its provider set
when the provider configuration changes. The server subscribes to
`AIProvidersChangedChannel` (published by the provider CRUD endpoints)
and forwards each event as a payload-free signal, plus one signal on
subscribe; the gateway calls `GetAIProviders` on each signal to rebuild
its pool. The aibridged API is bumped to v1.2.
Env-seeded providers don't need a signal: seeding finishes before coderd
serves the gateway connection, so the gateway's initial fetch already
reflects the seeded set.
## For reviewers
The change is split into two commits to make review easy:
1. **`feat: synchronise provider changes with WatchAIProviders`** is a
faithful cherry-pick of #26797, identical to the originally reviewed PR.
It is committed without pre-commit hooks because it does not build
against current `main` on its own.
2. **`fix: resolve cherry-pick conflicts against main`** contains only
the deltas needed to re-land on current `main`, and passes the full
pre-commit suite:
- `coderd/aibridged/proto/aibridged.pb.go` regenerated via the proto
make target (the cherry-picked copy was generated against the older
proto).
- `enterprise/cli/aigatewaystart.go` import block unioned; `main` added
`os` and `strings` while the PR added `sync`.
- Three `aibridgedserver.NewServer` test call sites that landed on
`main` after the original branch diverged now pass the new `pubsub`
argument.
Refs https://linear.app/codercom/issue/AIGOV-465
*This PR was produced by opencode (agent) using the
`anthropic/claude-opus-4-8` model, under human direction and review.*
Adds `/healthz` and `/readyz` endpoints to standalone AI Gateway.
* `/healthz`: returns 200 once the AI Gateways HTTP server is listening.
* `/readyz`: returns 200 when the DRPC connection to `coderd` is
established.
Cleanup:
Removed `initConnection*` fields from `aibridged.Server` as they where
not used anywhere.
> AI Tools were used to produce this PR
This PR adds `coder ai-gateway start` command that runs the AI Gateway
as an independent process.
- Standalone process doesn't have access to DB. Uses DRPC services under
`/api/v2/ai-gateway/serve`for auth, recording and provider
initialization.
- It only handles LLM traffic, other endpoints (eg. `/sessions`) are
only available though `coderd`.
- The standalone gateway reuses applicable flags from AI Gateway
deployment options. Provider-seeding and coderd-only options are
excluded.
- Only added to fat build, the slim build stub rejects the command.
Some wiring used by this new command is added.
**`NewWebsocketDialer`** - implements the standalone gateway's
connection to coderd's `/api/v2/ai-gateway/serve` endpoint. It upgrades
to a WebSocket, multiplexes with yamux, and wires all DRPC services.
**`AIGatewayDataPlaneMiddleware`** - extracts the per-request middleware
chain (concurrency limiting, rate limiting, BYOK gating) into a shared
function used by both the embedded route and the standalone gateway.
**`RootCmd.ResolveClientConnection`** - resolve the deployment URL and
builds an HTTP transport without requiring a session token. Used in
`ai-gateway start`command as it authenticates using different credential
type.
---------
Co-authored-by: Danny Kopping <danny@coder.com>
## Overview
Part of the **boundary correlation** feature. Fixes lazy creation of
`boundary_sessions` rows so it works within the agent's RBAC
constraints, and consumes the new `ConfinedProcessName` field reported
by boundary.
Pairs with coder/boundary#206, which adds `ConfinedProcessName` to
`ReportBoundaryLogsRequest`. This branch bumps the
`github.com/coder/boundary` module to pick up that work.
## Problem
`ensureSession` did a pre-insert existence check via
`GetBoundarySessionByID`. Agents are **not permitted to read boundary
sessions**, so that read path is not viable when the session is created
from an agent-reported log batch.
## Changes
- **Remove the pre-insert read.** `ensureSession` now inserts directly
and treats a primary-key unique violation as success, covering sessions
already created by a prior batch, a reconnection, or another coderd
replica — without requiring read access.
- **Per-connection guard.** Add a mutex-protected `ensuredSessions` set
so repeated log batches on the same connection skip the existence check
and insert entirely, touching the database only for the logs. On a
transient insert failure the session is left unmarked so the next batch
retries.
- **Consume `ConfinedProcessName`.** Pass `req.GetConfinedProcessName()`
through to the session insert.
- **Bump boundary module** from `v0.9.0` to
`v0.9.1-0.20260706095856-35ba90f9e8b2`.
- **Tests.**
- Add `TestReportBoundaryLogsAgentRBAC`
(`coderd/boundary_logs_test.go`), an integration test that connects as a
real workspace agent, verifies the session and log are persisted under
agent RBAC, and asserts the agent subject cannot read boundary sessions
— guarding against reintroducing a pre-insert read.
- Add `TestReportBoundaryLogsSessionGuard` (session inserted once across
two batches, logs inserted per batch) and
`TestReportBoundaryLogsSessionRetriedOnError` (insert retried after a
transient error).
- Regenerate `agent-firewall` CLI docs/golden files and adjust the
clidocgen template to render the YAML path when a flag has no long name.
> 🤖 This PR was opened by Coder Agents on behalf of @SasSwart.
Hides UI, CLI and API related to AI Gateway key management +
`/api/v2/ai-gateway/serve` endpoint.
API endpoints and CLI commands are still working they are just not
visible.
Rename user-facing "AI Bridge" strings to "AI Gateway" in deployment
config, RBAC display names, log messages, error strings, docs style
guide, and Grafana dashboard README.
Deprecated option names and descriptions (the `--aibridge-*` block) are
intentionally kept as "AI Bridge". The `Name` field cannot be renamed
because `serpent` uses it as a unique key during JSON serialization;
duplicating names causes `UnmarshalJSON` failures (e.g. in the support
bundle). Descriptions also stay as "AI Bridge" to avoid confusion
between the deprecated and primary options.
Refs https://linear.app/codercom/issue/AIGOV-226
> Generated with the assistance of Coder Agents (@ssncferreira)
Closes [AIGOV-455](https://linear.app/codercom/issue/AIGOV-455/extend-drpc-with-buildproviders).
## Why
The AI Gateway (`aibridged`) is being split into a standalone process that must not touch the database. `coderd` stays the source of truth and seeds the `ai_providers` / `ai_provider_keys` tables from the environment. This PR adds a DRPC call so the gateway fetches provider config from `coderd` instead of reading the DB, for both the embedded and standalone daemons.
## What
- **Proto:** new `ProviderConfigurator` service with a unary `GetAIProviders` RPC, plus `AIProvider` / `AIProviderBedrock` messages. `CurrentMinor` bumped to 1 (additive).
- **Server (`coderd/aibridgedserver`):** `GetAIProviders` runs a read-only `InTx` under `LockIDAIProvidersEnvSeed` so it never returns a mid-seed snapshot, reads providers (incl. disabled) plus keys for enabled ones, and maps to proto under `dbauthz.AsAIBridged`. Unmappable rows are skipped and logged; plaintext keys and Bedrock secrets are never logged.
- **Client:** `DRPCProviderConfiguratorClient` wired into the client union, `dialer.go`, and `CreateInMemoryAIBridgeServer`.
- **cli:** `BuildProvidersFromProto` maps the response through the existing DB-neutral `buildProvider`. A shared `poolRPCReloader` does the fetch/build/replace for both daemons: the embedded daemon reloads on every `ai_providers` change and fails startup if it cannot subscribe; the standalone gateway drives the same reloader once at startup, retrying until success and staying interruptible.
- **Dead code removed:** `BuildProvidersFromConfig`, `ProvidersFromConfig`, `AIProviderFromConfig`, and the DB-read `BuildProviders` path.
Renames the `last_used_at` column to `last_heartbeat_at` in `ai_gateway_keys` table.
`ai_gateway_keys` table has not been released yet.
All references updated.
Closes GRU-69
Adds CODER_CLUSTER_HOST enviroment variable and CLI arg.
I ended up not making it hidden since we'll just have to unhide it later and even when hidden it still shows up in some autogenerated stuff. Might as well just go for it.
I also added it to the helm chart.
relates to GRU-69
Modifies replicasync to handle discovering NATS enabled primary replicas explicitly, and passing that info to the NATS Pubsub.
This PR adds a new deployment value to explicitly represent the host or IP that the replica can be reached on. It isn't wired up to the CLI, but piggybacks on the DERP config for now.
We learn the NATS port directly from NATS at runtime, and propagate it thru replicasync to learn all peers for clustering.
Migrate `wg.Add(1); go func() { defer wg.Done(); ... }()` to
`wg.Go(func() { ... })` in tests.
Where the prior pattern passed the loop variable explicitly via a
closure parameter (`go func(id int) { ... }(i)`), drop the parameter and
reference the loop variable directly. Per-iteration loop variables since
Go 1.22 make this safe.
Part of the Template Builder wizard PR stack.
## Backend fixes
1. **Registry URL scheme fix**: Default
`CODER_TEMPLATE_BUILDER_REGISTRY_URL` was `https://registry.coder.com`
but Terraform module registry addresses must be scheme-less. Changed to
`registry.coder.com`.
2. **Sensitive variable defaults**: Module `.tf.tmpl` files for
claude-code, aider, amazon-q had sensitive `variable` blocks without
`default`, causing `terraform plan` to fail during template import. Also
fixed the `templatebuildermodulegen` script.
3. **Auto-quote string variables**: The backend now accepts raw string
values from callers and wraps them in HCL quotes automatically.
Previously callers were required to send pre-quoted HCL literals, which
is not a reasonable API contract.
---
> [!NOTE]
> Generated by Coder Agents on behalf of @jeremyruppel
Adds `coder ai-gateway keys` commands:
* `create <name>` creates key with given name
* `list` lists existing keys (alias `ls`)
* `delete <name | id>` removes key matching by name or key id, name has
priority (alias `rm`)
> AI Tools where used in this request.
Registers `coder_aibridged_*` and `coder_aibridgeproxyd_*` metrics under
new prefixes: `coder_ai_gateway_*` and `coder_ai_gateway_proxy_*`.
Old prefix is still exported. Will be removed in later release.
Also updated the `metricsdocgen` static fixture. Added 4
previously-undocumented metrics `key_pool_state`,
`key_pool_state_transitions_total`, `key_pool_exhaustions_total`,
`key_pool_failover_attempts` added the `client` label to the existing
interception, prompt, and token counter samples.
Updated AI Gateway documentation.
Add a periodic purge job for `boundary_logs` rows past their retention
threshold, following the same pattern as the existing audit log and
connection log purge jobs in `dbpurge`.
Expose a `--boundary-log-retention` deployment flag (env
`CODER_BOUNDARY_LOG_RETENTION`, YAML `retention.boundary_logs`). Default
is `0` (keep indefinitely). When set to a positive duration, `purgeTick`
deletes rows where `captured_at` is older than the threshold in batches
of 10,000, matching other log purge operations. The `boundary_logs`
label is added to the `records_purged_total` Prometheus counter.
Also removes the random-UUID fallback for `OwnerID` in
`dbgen.BoundarySession`. The previous fallback generated a UUID that
could never satisfy the `boundary_sessions_owner_id_fkey` FK constraint,
masking test setup bugs. Callers must now provide a valid user ID or
accept NULL (the legitimate "user deleted" state).
Adds `ai` to sqlc's `gen.go.initialisms` in `coderd/database/sqlc.yaml`
so the generated DB code follows Go's initialism convention. Adds the
matching `ai` -> `AI` case to the dbgen PascalCase helper
(`scripts/dbgen/main.go`) so the corresponding `dbmem` / mock
identifiers stay in sync. `make gen` regenerates the rest; hand-written
call sites that consume DB-generated identifiers
(`enterprise/audit/table.go`, `coderd/database/modelmethods.go`,
`enterprise/coderd/aigatewaykeys.go`, `coderd/database/dbauthz/*`, etc.)
are updated to match.
Scope is deliberately limited to the database layer:
- `coderd/rbac/*` (resource and scope generators) is untouched —
`ResourceAi*` / `ScopeAi*` constants stay on main's casing.
- `codersdk/*` (Go SDK) is untouched — `codersdk.ResourceAi*` /
`codersdk.APIKeyScopeAi*` constants stay on main's casing, so external
Go SDK consumers see no source-level break.
- `Aibridge*` identifiers (one SQL token `aibridge`, not `ai_bridge`)
are out of scope.
On-the-wire values are unchanged: enum strings, RBAC resource type
strings, API key scope strings, and JSON tags all stay the same. The
HTTP/JSON surface is unaffected.
Refs:
[AIGOV-369](https://linear.app/codercom/issue/AIGOV-369/change-ai-references-in-coderddatabasemodelsgo-to-ai)
🤖 Generated with [Coder Agents](https://coder.com)
## Summary
Removes the deprecated `/api/v2/aibridge/interceptions` endpoint and the
Request Logs frontend page, both replaced by the session-based view.
Closes https://linear.app/codercom/issue/AIGOV-266
Closes https://linear.app/codercom/issue/AIGOV-324
## Changes
### Backend
- Remove `GET /api/v2/aibridge/interceptions` HTTP handler and route
- Remove SDK types and client method (`AIBridgeInterception`,
`AIBridgeTokenUsage`, `AIBridgeUserPrompt`, `AIBridgeToolUsage`,
`AIBridgeListInterceptionsResponse`, `AIBridgeListInterceptionsFilter`)
- Remove SQL queries `CountAIBridgeInterceptions` and
`ListAIBridgeInterceptions`
- Remove `searchquery.AIBridgeInterceptions` parser
- Remove dbauthz wrappers, in-memory implementations, metrics, and mocks
for the interceptions list queries
- Remove the `coder aibridge interceptions list` CLI command and golden
files
- Regenerate API docs, swagger, mocks, and metrics
The `/models`, `/clients`, and `/sessions` endpoints stay; the sessions
list page still consumes all three.
### Frontend
- Delete the entire `RequestLogsPage/` directory (page, view, row,
filter, stories, tests)
- Remove the `/aibridge/request-logs` route and its lazy import
- Remove the `getAIBridgeInterceptions` API method,
`paginatedInterceptions` query, and mock interception entities
- `git mv` the shared filter and icon components used by the sessions
pages:
- `RequestLogsPage/RequestLogsFilter/{Client,Model,Provider}Filter.tsx`
→ `AIBridgePage/filters/`
- `RequestLogsPage/icons/AIBridge{Client,Model,Provider}Icon.tsx` →
`AIBridgePage/icons/`
- Drop the `getProviderIconName` hack and the duplicate `anthropic-neue`
icon case now that the FIXME no longer applies
## Commits
1. `refactor: remove interceptions API and request logs view` — the bulk
removal, with explicit renames for the shared filter/icon files.
2. `refactor(site/src/pages/AIBridgePage): drop getProviderIconName
hack` — cleanup of the FIXME that depended on RequestLogsPage existing.
> [!NOTE]
> Generated by Coder Agents on behalf of @dannykopping
Subdomain app routing derived the app identity from
httpapi.RequestHost, which returned the client-supplied
X-Forwarded-Host header verbatim. No middleware validated or stripped
that header, so a request from an untrusted peer could forge it. Since
the application_connect cookie is scoped to the wildcard apps domain,
JavaScript in a share=authenticated app could fetch() with a forged
X-Forwarded-Host pointing at a victim's owner-only app; coderd routed
and authorized the request as the victim and returned the private app
response same-origin to the attacker.
Replace RequestHost with httpmw.EffectiveHost, which honors
X-Forwarded-Host only when the original socket peer is a configured
trusted origin, otherwise falling back to the received Host header.
This ties host trust to the same RealIPConfig model already used for
X-Forwarded-For and -Proto. Wire it into HandleSubdomain for both
coderd and wsproxy, and log both the effective host and the raw
received_host.
Add coverage: EffectiveHost unit tests assert the trust decision uses
the socket peer rather than the spoofable forwarded client IP, and a
HandleSubdomain test confirms a forged X-Forwarded-Host from an
untrusted peer never reaches token resolution.
Refs: https://linear.app/codercom/issue/PLAT-259
Reverts coder/coder#26239
We cannot disable a feature which was previously enabled; this is a BC
break.
This is also using `AIGatewayRoutingEnabled` which will be removed in
the next release.
Problem: CODER_AI_GATEWAY_ENABLED defaulted to true, which both started
the in-memory gateway and enabled the licensed FeatureAIBridge. As a
result, deployments that never configured AI Gateway saw a spurious "AI
Governance add-on is required" warning whenever they had an older
(non-add-on) Premium license, since the feature was enabled-and-entitled
by default.
Fix: Decouple "external AI Gateway API enabled" from "in-memory daemon
running," so the external/licensed surface is off by default while Coder
Agents retain access by default.
Resolves the issue of `--prompt-ephemeral-parameters` and
`--ephemeral-parameter` not being available for use in the `coder
create` workspace creation command (they are only available in `coder
start` command). Back when they were [added
originally](https://github.com/coder/coder/pull/15030) it seems to have
been an oversight that they were left out.
The problem this solves:
```
coder create --parameter my_ephemeral_parameter=foo
error: prepare build: ephemeral parameter "my_ephemeral_parameter" can be used only with --prompt-ephemeral-parameters or --ephemeral-parameter flag
```
```
coder create my-test-ws -t general --ephemeral-parameter my_ephemeral_parameter=foo
parsing flags ([create my-test-ws -t general --ephemeral-parameter my_ephemeral_parameter=foo]) for "coder create": unknown flag: --ephemeral-parameter
```
Tested on a template with the following:
```
data "coder_parameter" "my_ephemeral_parameter" {
name = "my_ephemeral_parameter"
type = "bool"
description = "true or false?"
mutable = true
default = false
ephemeral = true
}
resource "coder_env" "debug_ephemeral" {
agent_id = coder_agent.main.id
name = "EPHEMERAL_TEST"
value = data.coder_parameter.my_ephemeral_parameter.value
}
```
By running:
```
➜ coder git:(rowan/coder-create-5495) ✗ go run cmd/coder/main.go create --ephemeral-parameter my_ephemeral_parameter=true
> Specify a name for your workspace: ws4
Select a template below to preview the provisioned infrastructure:
? kasmvnc-ubuntu-coder-dev used by 1 active developer
Select a preset below:
? Small (2 CPU / 4 GB)
....
...
The ws4 workspace has been created at Jun 3 12:36:38!
➜ coder git:(rowan/coder-create-5495) ✗ coder ssh ws4
workspace-ws4-5d6994756f-qlwnl% echo $EPHEMERAL_TEST
true
workspace-ws4-5d6994756f-qlwnl% exit
```
- Adds server-side and client-side validation for
CODER_CONFIGSSH_HOSTNAME_SUFFIX and CODER_SSH_CONFIG_OPTIONS.
- **Server-side breaking change:** invalid values for either of these will cause `coderd` to exit with an error.
- Client-side: `coder config-ssh` will exit with an error if it detects invalid config.
- Adds tests for the above
Local smoke-testing: ran `develop.sh --env-file <path to an env file
containing badness>`. Validated that server startup failed as expected.
> 🤖 Generated by Coder Agents with supervision from a human.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Renames the `coder boundary` CLI subcommand to `coder agent-firewall` as
part of the Boundaries → Agent Firewall rebrand.
`coder boundary` is retained as a hidden, deprecated alias that prints a
deprecation notice to stderr before running. Both commands use separate
builder functions backed by the same boundary base command and license
verification logic.
Closes https://linear.app/codercom/issue/AIGOV-236
<details><summary>Implementation notes</summary>
**Approach:** Two separate `*serpent.Command` objects (not `Aliases`) so
the deprecated `boundary` path can print a stderr warning while
`agent-firewall` stays clean.
**Changes:**
- `enterprise/cli/boundary.go`: Split old `boundary()` into
`buildAgentFirewallCmd()` and `buildBoundaryAliasCmd()`. Error messages
in `verifyLicense` now reference "agent-firewall".
- `enterprise/cli/root.go`: Register both commands.
- `cli/root.go`: Update YAML-only option validation bypass for the new
command name.
- Tests: Rename to `TestAgentFirewallSubcommand`, add
`TestBoundaryAlias`, update license verification tests to use
`agent-firewall`.
- Golden files and CLI reference docs regenerated.
- `docs/ai-coder/agent-firewall/version.md` and `docs/manifest.json`
updated.
</details>
> Generated with [Coder Agents](https://coder.com/agents) by @SasSwart
Cleans the last few instances of ExpectMatch that didn't use the new `(ctx, ...)` variant, then deletes the deprecated method and renames `ExpectMatchContext` to drop the `Context` suffix.
Adds an optional dbcrypt wrapper around gitsshkeys.private_key. The
column is encrypted on insert and update through enterprise/dbcrypt when
external token encryption is configured, and decrypted on read.
A new private_key_key_id column references
dbcrypt_keys(active_key_digest) so revocation safety is enforced by the
existing foreign key. Rows with a NULL key_id stay plaintext and remain
readable. Existing plaintext rows can be backfilled by running `coder
server dbcrypt rotate`.
Generated with assistance from Coder Agents.
## Summary
Fixes all 9 Windows CI test failures caused by the mise CI refactor
(`fe257666d7`, PR #25727).
### Root cause
`jdx/mise-action` exports `Path` (Windows convention) via `GITHUB_ENV`.
Bash on Windows maintains its own `PATH`. When Go's `os.Environ()`
returns both, `cmd.exe` subprocesses non-deterministically pick the
MSYS-translated `PATH` (forward slashes), causing Windows executables
(`printf`, `powershell.exe`, `cmd.exe`) to be unresolvable.
These failures only appeared on `main` (where `-count=1` forces real
test execution) and were masked on PRs by Go test cache.
### Fixes applied
**CI (`setup-mise` action)**:
- Write both `Path` and `PATH` to `GITHUB_ENV` with Git usr/bin
prepended
**Code (`cli/root.go`)**:
- Add `appendAndDedupEnv` helper that deduplicates case-insensitive env
vars on Windows, preferring native Windows paths (backslashes) over MSYS
paths
**Code (`cli/configssh_windows.go`)**:
- Use absolute paths for `powershell.exe` and `cmd.exe` in the SSH
config `Match exec` escape function, avoiding PATH resolution entirely
**Tests**:
- Switch `--header-command` tests from `printf` to `echo` (cmd.exe
builtin) for reliable cross-platform execution
- Add env dedup in `Test_sshConfigMatchExecEscape` for subprocess PATH
consistency
Fixescoder/internal#1556, coder/internal#1558, coder/internal#1559
> 🤖 Generated by Coder agent, will be reviewed by @mafredri. 🏂🏻
Add metrics for `aibridged` and `aibridgeproxyd`'s provider statuses. AI providers can be modified, and possibly misconfigured, at runtime. These metrics help operators understand the state of these provider definitions in case unexpected behaviour is observed.
Refactors the `aibridgeproxyd` provider reload mechanism which was unnecessarily complex.
Also ensures that providers are evaluated on each CONNECT request to prevent interception of requests to (newly) disabled providers; in this case the requests will passthrough unencrypted, by design.
Fake agents now fetch their manifest, spawn a single per-agent metadata
goroutine, and emit batched BatchUpdateMetadata calls with 3072-byte
base64 payloads so scaletest runs mirror the load shape of real agents.
This matches what the current scaletest workspace template does for
metadata. In the future we can extend the harness here to take in a
config option for the metadata payload size.
---------
Signed-off-by: Callum Styan <callumstyan@gmail.com>
Co-authored-by: Mux <mux@coder.com>
Previously the in-process aibridge daemon and the enterprise aibridgeproxy daemon both snapshotted their provider routing once at boot. Any `ai_providers` or `ai_provider_keys` mutation required a restart for either to pick it up.
Add an `ai_providers_changed` pubsub channel that the CRUD handlers publish on after Create / Update / Delete. Both daemons subscribe:
- **aibridged** rebuilds its `[]aibridge.Provider` snapshot via `BuildProviders` and swaps it into the pool atomically. Inflight requests keep serving against the bridge they already acquired; new acquires build against the new snapshot. Per-provider construction errors stay scoped to the offending row.
- **aibridgeproxyd** rebuilds its routing snapshot from `GetAIProviders` and swaps the host→provider map atomically. The MITM listener picks up new providers without restart.
DB read for aibridgeproxyd uses the existing `AsAIProviderMetadataReader` subject for routing-only access.
Adds an end-to-end enterprise CLI test to ensure legacy AI provider keys seeded at server startup are encrypted at rest when DBCrypt external token encryption is enabled, preventing regressions related to #25699.
> Partially implemented by Coder Agents, and massaged afterwards by me.
## Problem
Two related symptoms of the same architectural issue: the `dbcrypt`
wrapper is installed inside `enterprise/coderd.New`, so any access to
`options.Database` that happens before `newAPI` runs bypasses
encryption.
**Symptom 1 (reads):** Provider keys added via the admin UI are
encrypted at rest. `BuildProviders` was running *before* `newAPI`,
against the unwrapped store, so the ciphertext was read as-is and shoved
into the keypool as the upstream credential. Anthropic/OpenAI reject it,
and the interception log shows:
```
coderd.aibridged.pool: interception failed ... error="all configured keys failed authentication"
credential_kind=centralized credential_hint=PaPb...4A== credential_length=184
```
**Symptom 2 (writes):** `SeedAIProvidersFromEnv` was also running before
`newAPI`, against the unwrapped store, so env-derived keys
(`CODER_AIBRIDGE_OPENAI_KEY`, indexed `CODER_AIBRIDGE_PROVIDER_<N>_KEY`,
etc.) landed in `ai_provider_keys` as plaintext with `ApiKeyKeyID =
null` even when `CODER_EXTERNAL_TOKEN_ENCRYPTION_KEYS` was set.
## Fix
Move both `SeedAIProvidersFromEnv` and `BuildProviders` to after
`newAPI`, where `options.Database` is the dbcrypt-wrapped store. Writes
encrypt correctly; reads decrypt correctly.
The enterprise closure (`enterprise/cli/server.go`) runs *inside*
`newAPI` and calls `BuildProviders` for the aibridgeproxyd at that
point. Once the agpl seed moves to after `newAPI`, the proxy on first
boot would see no env-seeded providers. Add a matching seed call inside
the enterprise closure before its `BuildProviders` to cover that case.
Seeding is idempotent, so the agpl-side seed running again post-`newAPI`
is a no-op when the rows already exist.
## Known shortcomings
The clean version of this fix would just inherit `ctx` like every other
startup step and place these calls naturally. It can't, for two reasons
that are both about the surrounding handler architecture rather than
this change:
1. **`dbcrypt` wrapping is positioned inside `newAPI`, not around
`options.Database` at creation.** That's why both seed and build have to
wait until after `newAPI` in the first place. The principled fix is to
install the wrapper at the point the store is created (behind a hook the
enterprise build supplies), so every consumer sees a single
authoritative view and the ordering stops mattering. This would also
collapse the duplicated seed call back to a single site.
2. **The handler's shutdown sequence is not deferred.**
`coderAPICloser.Close()` and the other teardown steps run only if
control reaches the `select` at the bottom of the handler. An early
`return` from anywhere in Phase 1 (e.g. seed/build returning
`context.Canceled` when the user hits ctrl-c during startup) skips that
block and orphans all the goroutines `newAPI` spawned — tailnet workers,
gitsync, telemetry batcher, etc. `goleak` then catches them at package
teardown and `TestServer_TelemetryDisabled_FinalReport` fails. Moving
the shutdown into deferred closers (with a `sync.Once`-guarded close to
avoid double-close from the explicit Phase 2 call) is the principled
fix.
For this PR I took the smallest change that fixes the reported bugs: a
detached context (`context.WithoutCancel(ctx)` + a 30s timeout) at the
seed and build call sites in both the agpl and enterprise paths. It lets
the calls complete even if the user cancels during startup, after which
the handler reaches its shutdown select naturally and tears down through
Phase 2. Both shortcomings above are worth addressing separately.
## Test plan
- `make test RUN=TestServer_TelemetryDisabled_FinalReport` with `-race`;
passes locally with `-count=3`.
- Manually verified on a deployment with
`CODER_EXTERNAL_TOKEN_ENCRYPTION_KEYS` set and env-configured providers:
`ai_provider_keys.api_key_key_id` is populated, `api_key` is base64
ciphertext, and upstream auth succeeds.
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace the env-based `BuildProviders` with a DB-backed loader. The database is now the single source of truth for runtime provider configuration; env config arrives via `SeedAIProvidersFromEnv` (run at boot) and `BuildProviders` reads it back as `aibridge.Provider` instances. `cli/server.go` and `enterprise/cli/server.go` both call the same path, so aibridged and aibridgeproxyd see the same provider set.
Per-provider `DumpDir` is replaced by a top-level `CODER_AI_GATEWAY_DUMP_DIR` base; each provider's effective dump path is `<base>/<provider name>`.
In order to allow Coder Agents to use AI Gateway in OSS, we need to rehome the `aibridged`\-related code into the AGPL path.
The HTTP API is only registered under enterprise so will still require the AI Governance Add-on to be present in order to use it, whereas Coder Agents uses an in-memory pipe to the same handlers.
`CODER_AI_GATEWAY_ENABLED` / `CODER_AIBRIDGE_ENABLED` is now being defaulted to `true` now that it will be used by Coder Agents.
If you previously had this value disabled explicitly, that value will persist.
_Disclaimer: implemented by a Coder Agent using Claude Opus 4.7_
Part of the implementation of [RFC: Common AI Provider Configs](https://www.notion.so/coderhq/RFC-Common-AI-Provider-Configs-34bd579be59280ed958feffb82024797) (AIGOV-201).
## Note
This change can cause a previously working installation to fail to start should a conflict exist between the providers configured in the environment & those now migrated to the database.
I'll raise a PR upstack to document this process and workarounds should a startup fail.
## What this PR does
Reconciles environment-derived AI provider configuration with the `ai_providers` table at server startup. The seed runs **before** the aibridged daemon is initialized, so the runtime always reads providers from the database; the legacy `CODER_AIBRIDGE_*` environment variables become a one-shot migration source.
### Behavior
- Concurrent server starts are serialized through a Postgres advisory lock (`LockIDAIProvidersEnvSeed`).
- Missing rows are inserted with an audit entry attributed to the system actor.
- Existing rows whose canonical hash matches the env-derived hash are left alone (the common no-op restart path).
- Existing rows whose canonical hash does **not** match cause server startup to fail with a descriptive error so the operator can explicitly resolve the conflict in either env or DB.
- Soft-deleted rows are NOT resurrected from env; an explicit operator deletion is sticky across restarts.
- Indexed providers whose name conflicts with a legacy env var fail startup with a clear remediation message.
- Unknown provider types (e.g. `copilot`, until the DB enum is widened) are skipped with a log entry rather than failing startup.
### Canonical hashing
The `canonicalAIProvider` shape captures exactly the fields that determine runtime behavior — `type`, `base_url`, and the Bedrock subset of settings (access key, access key secret, region, model, small fast model) — and is hashed with SHA-256. The hash is **computed on demand from the row + env**, never persisted, so the database does not need a new column for it. API keys live in the separate `ai_provider_keys` table and are intentionally excluded from the hash so operators can rotate keys via the API without forcing a server restart.
<details>
<summary>Decision log</summary>
- The hash is intentionally not persisted in the database. The RFC discussed this trade-off; computing on demand keeps the schema minimal and lets the canonical shape evolve without a migration.
- The lock uses an `iota` slot in `coderd/database/lock.go` rather than `GenLockID` so it's stable, easy to audit, and matches the convention used for every other startup lock.
- A bearer-token Anthropic provider whose env vars also set Bedrock metadata but no AWS credentials does NOT store the Bedrock fields. Without credentials the discriminated settings would misrepresent the row as Bedrock auth.
- We deliberately do NOT publish to the `ai_providers_changed` pubsub channel from the seed because the seed completes before any subscriber is started; the follow-up PR introduces that channel.
</details>
Paweł Banaszewski