mirror of
https://github.com/coder/coder.git
synced 2026-09-24 15:04:27 +08:00
fix: support Bedrock ambient AWS credentials for Agents providers (#24397)
> This PR was authored by Mux on behalf of Mike. Adds AWS Bedrock ambient credential support to the Agents provider path. Bedrock providers can now be saved without a stored API key and authenticated via the standard AWS SDK credential chain on the Coder server (IAM roles, `AWS_ACCESS_KEY_ID`, etc.). Also fixes missing `Base URL` forwarding for Bedrock. ## Changes **Backend runtime** (`coderd/x/chatd/chatprovider/chatprovider.go`): - New `ProviderAllowsAmbientCredentials(provider)` helper. Currently returns true only for Bedrock. - `ModelFromConfig` no longer errors on an empty API key when the provider is in the ambient-allowed set AND was explicitly resolved via `ByProvider`. This preserves the policy gate: unresolvable providers (disabled central key, user-key-required without a user key) still error. - `setResolvedProviderAPIKey` internalizes the ambient-credentials contract via `ProviderAllowsAmbientCredentials`, so a resolved-but-keyless Bedrock provider is represented as an empty `ByProvider` entry rather than a post-hoc sentinel patch in the caller. - `WithAPIKey` is only appended when a token is present. - `WithBaseURL(baseURL)` is now forwarded for Bedrock (was previously missing). **Backend admin API** (`coderd/exp_chats.go`): - `validateChatProviderCentralAPIKey` exempts Bedrock from requiring a stored API key when central credentials are enabled. - AI Gateway separation (`ChatProviderAPIKeysFromDeploymentValues`) is unchanged. No silent reuse of `CODER_AIBRIDGE_BEDROCK_*` flags. **Frontend** (`site/src/pages/AgentsPage/components/ChatModelAdminPanel/*`): - API Key field is optional for Bedrock when central credentials are enabled. - Bedrock-specific descriptions on API Key and Base URL fields (bearer-token vs ambient modes, `AWS_REGION` guidance). - Right-aligned "Clear stored token" action switches an existing Bedrock provider back to ambient mode. - `hasEffectiveAPIKey` treats Bedrock with central credentials enabled as configured, so the provider list shows the correct status icon. - Three new stories: `ProviderFormBedrockAmbientCredentials`, `ProviderFormBedrockBearerToken`, `ProviderFormBedrockClearBearerToken`. **Docs** (`docs/ai-coder/agents/models.md`, `docs/ai-coder/ai-gateway/setup.md`): - New "Configuring AWS Bedrock" section covering both credential modes, region resolution, and the Base URL override. - Explicit note that the `us-east-1` region fallback only applies to bearer-token mode; ambient credentials require a region from the standard AWS SDK chain. - Cross-reference in AI Gateway docs clarifying that `CODER_AIBRIDGE_BEDROCK_*` flags are a separate configuration path from Agents. ## Not in scope - Reusing AI Gateway Bedrock flags as an implicit Agents fallback. - Per-provider AWS access key, secret, or region fields (would need a migration and audit-table review). - IMDS or network-backed credential probes in admin/listing request paths. ## Related Dogfood deployment integration: https://github.com/coder/dogfood/pull/324
This commit is contained in:
Michael Suchacz
@@ -1,29 +1,29 @@
|
||||
# Models
|
||||
|
||||
Administrators configure LLM providers and models from the Coder dashboard.
|
||||
Providers, models, and API keys are deployment-wide settings managed by
|
||||
platform teams. Developers select from the set of models that an administrator
|
||||
has enabled.
|
||||
Providers, models, and centrally managed credentials are deployment-wide
|
||||
settings managed by platform teams. Developers select from the set of models
|
||||
that an administrator has enabled.
|
||||
|
||||
Optionally, administrators can allow developers to supply their own API keys
|
||||
for specific providers. See [User API keys](#user-api-keys-byok) below.
|
||||
|
||||
## Providers
|
||||
|
||||
Each LLM provider has a type, an API key, and an optional base URL override.
|
||||
Each LLM provider has a type, a credential configuration, and an optional base URL override.
|
||||
|
||||
Coder supports the following provider types:
|
||||
|
||||
| Provider | Description |
|
||||
|-------------------|------------------------------------------|
|
||||
| Anthropic | Claude models via Anthropic API |
|
||||
| OpenAI | GPT and o-series models via OpenAI API |
|
||||
| Google | Gemini models via Google AI API |
|
||||
| Azure OpenAI | OpenAI models hosted on Azure |
|
||||
| AWS Bedrock | Models available through AWS Bedrock |
|
||||
| OpenAI Compatible | Any endpoint implementing the OpenAI API |
|
||||
| OpenRouter | Multi-model routing via OpenRouter |
|
||||
| Vercel AI Gateway | Models via Vercel AI SDK |
|
||||
| Provider | Description |
|
||||
|-------------------|------------------------------------------------------------------|
|
||||
| Anthropic | Claude models via Anthropic API |
|
||||
| OpenAI | GPT and o-series models via OpenAI API |
|
||||
| Google | Gemini models via Google AI API |
|
||||
| Azure OpenAI | OpenAI models hosted on Azure |
|
||||
| AWS Bedrock | Models via AWS Bedrock (bearer token or ambient AWS credentials) |
|
||||
| OpenAI Compatible | Any endpoint implementing the OpenAI API |
|
||||
| OpenRouter | Multi-model routing via OpenRouter |
|
||||
| Vercel AI Gateway | Models via Vercel AI SDK |
|
||||
|
||||
The **OpenAI Compatible** type is a catch-all for any service that exposes an
|
||||
OpenAI-compatible chat completions endpoint. Use it to connect to self-hosted
|
||||
@@ -35,7 +35,7 @@ models, internal gateways, or third-party proxies like LiteLLM.
|
||||
1. Click **Admin** in the top bar to open the configuration dialog.
|
||||
1. Select the **Providers** tab.
|
||||
1. Click the provider you want to configure.
|
||||
1. Enter the **API key** for the provider.
|
||||
1. Enter the **API key** for the provider, if required.
|
||||
1. Optionally set a **Base URL** to override the default endpoint. This is
|
||||
useful for enterprise proxies, regional endpoints, or self-hosted models.
|
||||
1. Click **Save**.
|
||||
@@ -47,28 +47,58 @@ status.</small>
|
||||
|
||||
<img src="../../images/guides/ai-agents/models-add-provider.png" alt="Screenshot of the add provider form">
|
||||
|
||||
<small>Adding a provider requires an API key. The base URL is optional.</small>
|
||||
<small>Adding a provider usually requires an API key. AWS Bedrock can also use
|
||||
ambient AWS credentials. The base URL is optional.</small>
|
||||
|
||||
### Provider API keys and security
|
||||
## Configuring AWS Bedrock
|
||||
|
||||
Provider API keys are stored encrypted in the Coder database. They are never
|
||||
exposed to workspaces, developers, or the browser after initial entry. The
|
||||
dashboard shows only whether a key is set, not the key itself.
|
||||
AWS Bedrock supports two credential modes for Agents providers:
|
||||
|
||||
- **Bearer token mode**: Enter a Bedrock-compatible bearer token in the
|
||||
**API key** field when you add the provider.
|
||||
- **Ambient AWS credentials mode**: Leave the **API key** field empty. The
|
||||
Coder server resolves credentials from the standard AWS SDK credential chain,
|
||||
including IAM instance roles and `AWS_ACCESS_KEY_ID` /
|
||||
`AWS_SECRET_ACCESS_KEY` environment variables.
|
||||
|
||||
Region comes from the standard AWS SDK configuration. In most deployments, set
|
||||
`AWS_REGION` on the Coder server. Bearer token mode falls back to `us-east-1`
|
||||
when no region is configured. Ambient credentials require a region from the
|
||||
standard AWS SDK chain, for example `AWS_REGION`.
|
||||
|
||||
The **Base URL** field overrides the Bedrock runtime endpoint. Use it for
|
||||
custom endpoints or VPC endpoints.
|
||||
|
||||
> [!NOTE]
|
||||
> Agents Bedrock provider configuration is separate from AI Gateway Bedrock
|
||||
> flags (`CODER_AIBRIDGE_BEDROCK_*`). AI Gateway and Agents use independent
|
||||
> credential paths.
|
||||
|
||||
## Provider credentials and security
|
||||
|
||||
Provider API keys entered in the dashboard are stored encrypted in the Coder
|
||||
database. They are never exposed to workspaces, developers, or the browser
|
||||
after initial entry. The dashboard shows only whether a key is set, not the
|
||||
key itself.
|
||||
|
||||
When a provider uses ambient credentials, Coder resolves them from the server
|
||||
environment at request time instead of storing a secret in the database.
|
||||
|
||||
Because the agent loop runs in the control plane, workspaces never need direct
|
||||
access to LLM providers. See
|
||||
[Architecture](./architecture.md#no-api-keys-in-workspaces) for details
|
||||
on this security model.
|
||||
|
||||
### Key policy
|
||||
## Key policy
|
||||
|
||||
Each provider has three policy flags that control how API keys are sourced:
|
||||
Each provider has three policy flags that control how provider credentials are
|
||||
sourced:
|
||||
|
||||
| Setting | Default | Description |
|
||||
|-------------------------|---------|-----------------------------------------------------------------------------------------------------|
|
||||
| Central API key | On | The provider uses a deployment-managed API key entered by an administrator. |
|
||||
| Allow user API keys | Off | Developers may supply their own API key for this provider. |
|
||||
| Central key as fallback | Off | When user keys are allowed, fall back to the central key if a developer has not set a personal key. |
|
||||
| Setting | Default | Description |
|
||||
|-------------------------|---------|--------------------------------------------------------------------------------------------------------------------------|
|
||||
| Central API key | On | The provider uses deployment-managed credentials configured by an administrator. For most providers, this is an API key. |
|
||||
| Allow user API keys | Off | Developers may supply their own API key for this provider. |
|
||||
| Central key as fallback | Off | When user keys are allowed, fall back to deployment-managed credentials if a developer has not set a personal key. |
|
||||
|
||||
At least one credential source must be enabled. These settings appear in the
|
||||
provider configuration form under **Key policy**.
|
||||
@@ -87,10 +117,11 @@ to a given developer:
|
||||
| On | On | On | No | Uses central key |
|
||||
|
||||
When a developer's personal key is present, it always takes precedence over
|
||||
the central key. When user keys are required and fallback is disabled,
|
||||
the provider is unavailable to developers who have not saved a personal key —
|
||||
even if a central key exists. This is intentional: it enforces that each
|
||||
developer authenticates with their own credentials.
|
||||
deployment-managed credentials. When user keys are required and fallback is
|
||||
disabled, the provider is unavailable to developers who have not saved a
|
||||
personal key, even if deployment-managed credentials exist. This is
|
||||
intentional: it enforces that each developer authenticates with their own
|
||||
credentials.
|
||||
|
||||
## Models
|
||||
|
||||
@@ -196,14 +227,15 @@ fields appear dynamically in the admin UI when you select a provider.
|
||||
|
||||
> [!NOTE]
|
||||
> Azure OpenAI uses the same options as OpenAI. AWS Bedrock uses the same
|
||||
> options as Anthropic.
|
||||
> model configuration options as Anthropic (thinking budget, reasoning
|
||||
> effort).
|
||||
|
||||
## How developers select models
|
||||
|
||||
Developers see a model selector dropdown when starting or continuing a chat on
|
||||
the Agents page. The selector shows only models from providers that have valid
|
||||
API keys configured. Models are grouped by provider if multiple providers are
|
||||
active.
|
||||
credentials configured. Models are grouped by provider if multiple providers
|
||||
are active.
|
||||
|
||||
The model selector uses the following precedence to pre-select a model:
|
||||
|
||||
@@ -232,17 +264,17 @@ developers can supply their own API key from the Agents settings page.
|
||||
1. Enter your API key and click **Save**.
|
||||
|
||||
Personal API keys are encrypted at rest using the same database encryption
|
||||
as deployment-managed keys. The dashboard never displays a saved key — only
|
||||
whether one is set.
|
||||
used for deployment-managed provider secrets. The dashboard never displays a
|
||||
saved key, only whether one is set.
|
||||
|
||||
### How key selection works
|
||||
|
||||
When you start a chat, the control plane resolves which API key to use for
|
||||
each provider:
|
||||
When you start a chat, the control plane resolves which credential source to
|
||||
use for each provider:
|
||||
|
||||
1. If you have a personal key for the provider, it is used.
|
||||
1. If you do not have a personal key and central key fallback is enabled,
|
||||
the deployment-managed key is used.
|
||||
deployment-managed credentials are used.
|
||||
1. If you do not have a personal key and fallback is disabled, the provider
|
||||
is unavailable to you. Models from that provider will not appear in the
|
||||
model selector.
|
||||
@@ -251,8 +283,8 @@ each provider:
|
||||
|
||||
Click **Remove** on the provider card in the API Keys settings tab. If
|
||||
central key fallback is enabled, subsequent requests will use the shared
|
||||
deployment key. If fallback is disabled, the provider becomes unavailable
|
||||
until you add a new personal key.
|
||||
deployment-managed credentials. If fallback is disabled, the provider becomes
|
||||
unavailable until you add a new personal key.
|
||||
|
||||
## Using an LLM proxy
|
||||
|
||||
|
||||
@@ -61,6 +61,10 @@ If both are set, `CODER_AIBRIDGE_BEDROCK_BASE_URL` takes precedence.
|
||||
- `CODER_AIBRIDGE_BEDROCK_MODEL` or `--aibridge-bedrock-model`
|
||||
- `CODER_AIBRIDGE_BEDROCK_SMALL_FAST_MODEL` or `--aibridge-bedrock-small-fast-model`
|
||||
|
||||
> [!NOTE]
|
||||
> These Bedrock settings configure AI Gateway only. To configure Bedrock as an
|
||||
> Agents provider, see [Configuring AWS Bedrock](../agents/models.md#configuring-aws-bedrock).
|
||||
|
||||
**Optional:**
|
||||
|
||||
- `CODER_AIBRIDGE_BEDROCK_ACCESS_KEY` or `--aibridge-bedrock-access-key`
|
||||
|
||||
Reference in New Issue
Block a user