mirror of
https://github.com/coder/coder.git
synced 2026-09-24 15:04:27 +08:00
docs: clarify Coder Agents AI Gateway routing (#25852)
> Mux updated this PR on behalf of Mike. Clarifies that Coder Agents route through AI Gateway automatically, while admins configure Agents providers with upstream provider or proxy endpoint/base URLs. Moves Agents-specific setup and credential guidance into the Agents models page, removes the obsolete AI Gateway Coder Agents client page, removes Coder Agents from the AI Gateway external client list, and links BYOK credential selection to the global AI Gateway BYOK setting.
This commit is contained in:
Michael Suchacz
@@ -37,11 +37,13 @@ Before you begin, confirm the following:
|
||||
|
||||
To configure Coder Agents:
|
||||
|
||||
1. Navigate to the **Agents** page in the Coder dashboard.
|
||||
1. Open **Settings** > **Manage Agents** and select the **Providers** tab.
|
||||
Pick a provider, enter your API key, and save.
|
||||
1. Switch to the **Models** tab, click **Add**, and configure at least one
|
||||
model with its identifier, display name, and context limit.
|
||||
1. Navigate to **Admin settings** > **AI** and select **Providers**.
|
||||
1. Add or update a provider with its credentials and upstream endpoint, then
|
||||
save it.
|
||||
1. Navigate to the **Agents** page, open **Settings** > **Manage Agents**, and
|
||||
select **Models**.
|
||||
1. Click **Add** and configure at least one model with its identifier, display
|
||||
name, and context limit.
|
||||
1. Click the **star icon** next to a model to set it as the default.
|
||||
|
||||
Detailed instructions for each provider and model option are in the
|
||||
|
||||
+101
-121
@@ -1,77 +1,87 @@
|
||||
# Models
|
||||
|
||||
Administrators configure LLM providers and models from the Coder dashboard.
|
||||
Providers, models, and centrally managed credentials are deployment-wide
|
||||
settings managed by platform teams. Developers select from the set of models
|
||||
that an administrator has enabled.
|
||||
Administrators configure LLM providers from **Admin settings** > **AI** and
|
||||
Coder Agents models from the **Agents** settings page. Providers, models, and
|
||||
centrally managed credentials are deployment-wide settings managed by platform
|
||||
teams. Developers select from the set of models that an administrator has
|
||||
enabled.
|
||||
|
||||
Optionally, administrators can allow developers to supply their own API keys
|
||||
for specific providers. See [User API keys](#user-api-keys-byok) below.
|
||||
Optionally, administrators can enable AI Gateway Bring Your Own Key (BYOK)
|
||||
so developers can supply personal API keys for providers. See
|
||||
[User API keys](#user-api-keys-byok) below.
|
||||
|
||||
## Providers
|
||||
|
||||
Each LLM provider has a type, a credential configuration, and an optional base URL override.
|
||||
Each LLM provider has a type, credentials, and an endpoint/base URL for the
|
||||
upstream provider or proxy.
|
||||
|
||||
Coder supports the following provider types:
|
||||
|
||||
| Provider | Description |
|
||||
|-------------------|------------------------------------------------------------------|
|
||||
| Anthropic | Claude models via Anthropic API |
|
||||
| OpenAI | GPT and o-series models via OpenAI API |
|
||||
| Google | Gemini models via Google AI API |
|
||||
| Azure OpenAI | OpenAI models hosted on Azure |
|
||||
| AWS Bedrock | Models via AWS Bedrock (bearer token or ambient AWS credentials) |
|
||||
| OpenAI Compatible | Any endpoint implementing the OpenAI API |
|
||||
| OpenRouter | Multi-model routing via OpenRouter |
|
||||
| Vercel AI Gateway | Models via Vercel AI SDK |
|
||||
| Provider | Description |
|
||||
|-------------------|------------------------------------------|
|
||||
| Anthropic | Claude models via Anthropic API |
|
||||
| OpenAI | GPT and o-series models via OpenAI API |
|
||||
| Google | Gemini models via Google AI API |
|
||||
| Azure OpenAI | OpenAI models hosted on Azure |
|
||||
| AWS Bedrock | Models via AWS Bedrock |
|
||||
| OpenAI Compatible | Any endpoint implementing the OpenAI API |
|
||||
| OpenRouter | Multi-model routing via OpenRouter |
|
||||
| Vercel AI Gateway | Models via Vercel AI SDK |
|
||||
|
||||
The **OpenAI Compatible** type is a catch-all for any service that exposes an
|
||||
OpenAI-compatible chat completions endpoint. Use it to connect to self-hosted
|
||||
models, internal gateways, or third-party proxies like LiteLLM.
|
||||
|
||||
Coder Agents route model requests through AI Gateway automatically by using
|
||||
the provider configuration stored in Coder's database.
|
||||
|
||||
### Add a provider
|
||||
|
||||
1. Navigate to the **Agents** page in the Coder dashboard.
|
||||
1. Open **Settings** > **Manage Agents** and select the **Providers** tab.
|
||||
1. Click the provider you want to configure.
|
||||
1. Enter the **API key** for the provider, if required.
|
||||
1. Optionally set a **Base URL** to override the default endpoint. This is
|
||||
useful for enterprise proxies, regional endpoints, or self-hosted models.
|
||||
LLM providers are managed from the deployment AI settings, not from the Agents
|
||||
settings page.
|
||||
|
||||
1. Navigate to **Admin settings** > **AI**.
|
||||
1. Select **Providers**.
|
||||
1. Click **Add provider**.
|
||||
1. Select the provider type.
|
||||
1. Enter a unique lowercase provider name, the credentials, and the upstream
|
||||
provider or proxy
|
||||
[endpoint/base URL](#endpointbase-url-for-openai-compatible-providers).
|
||||
1. Click **Save**.
|
||||
|
||||
<img src="../../images/guides/ai-agents/models-providers.png" alt="Screenshot of the providers list in the Agents settings">
|
||||
After saving a provider, add an Agents model for it from **Agents** >
|
||||
**Settings** > **Manage Agents** > **Models**. For provider-specific setup,
|
||||
including AWS Bedrock, see
|
||||
[AI Gateway provider configuration](../ai-gateway/providers.md#provider-types).
|
||||
|
||||
<small>The providers list shows all supported providers and their configuration
|
||||
status.</small>
|
||||
## Endpoint/base URL for OpenAI-compatible providers
|
||||
|
||||
<img src="../../images/guides/ai-agents/models-add-provider.png" alt="Screenshot of the add provider form">
|
||||
Provider configuration stores an absolute HTTP(S) endpoint/base URL. Syntax
|
||||
validation confirms that the value is a URL, but it does not prove the upstream
|
||||
implements the APIs Coder sends.
|
||||
|
||||
<small>Adding a provider usually requires an API key. AWS Bedrock can also use
|
||||
ambient AWS credentials. The base URL is optional.</small>
|
||||
For the default Agents path through AI Gateway, set the endpoint/base URL to
|
||||
the upstream provider or proxy endpoint. Do not set it to Coder's public AI
|
||||
Gateway route, such as `https://<coder-host>/api/v2/aibridge/openai/v1`.
|
||||
|
||||
## Configuring AWS Bedrock
|
||||
OpenAI-shaped provider types require the upstream OpenAI-compatible prefix in
|
||||
the endpoint/base URL because Coder appends request suffixes such as
|
||||
`/chat/completions`, `/responses`, and `/models`. This applies to **OpenAI**,
|
||||
**Azure OpenAI**, **Google**, **OpenAI Compatible**, **OpenRouter**, and
|
||||
**Vercel AI Gateway** provider types.
|
||||
|
||||
AWS Bedrock supports two credential modes for Agents providers:
|
||||
Examples:
|
||||
|
||||
- **Bearer token mode**: Enter a Bedrock-compatible bearer token in the
|
||||
**API key** field when you add the provider.
|
||||
- **Ambient AWS credentials mode**: Leave the **API key** field empty. The
|
||||
Coder server resolves credentials from the standard AWS SDK credential chain,
|
||||
including IAM instance roles and `AWS_ACCESS_KEY_ID` /
|
||||
`AWS_SECRET_ACCESS_KEY` environment variables.
|
||||
| Provider type | Example endpoint/base URL |
|
||||
|-------------------------------------|------------------------------------------------------------|
|
||||
| OpenAI | `https://api.openai.com/v1/` |
|
||||
| Azure OpenAI | `https://<resource-name>.openai.azure.com/openai/v1` |
|
||||
| Google Gemini OpenAI-compatible API | `https://generativelanguage.googleapis.com/v1beta/openai/` |
|
||||
| OpenRouter | `https://openrouter.ai/api/v1` |
|
||||
| Vercel AI Gateway | `https://ai-gateway.vercel.sh/v1` |
|
||||
| Generic OpenAI-compatible proxy | `https://provider.example.com/v1` |
|
||||
|
||||
Region comes from the standard AWS SDK configuration. In most deployments, set
|
||||
`AWS_REGION` on the Coder server. Bearer token mode falls back to `us-east-1`
|
||||
when no region is configured. Ambient credentials require a region from the
|
||||
standard AWS SDK chain, for example `AWS_REGION`.
|
||||
|
||||
The **Base URL** field overrides the Bedrock runtime endpoint. Use it for
|
||||
custom endpoints or VPC endpoints.
|
||||
|
||||
> [!NOTE]
|
||||
> Agents Bedrock provider configuration is separate from AI Gateway Bedrock
|
||||
> flags (`CODER_AI_GATEWAY_BEDROCK_*`). AI Gateway and Agents use independent
|
||||
> credential paths.
|
||||
Confirm the exact endpoint/base URL in your provider or proxy documentation.
|
||||
|
||||
## Provider credentials and security
|
||||
|
||||
@@ -80,47 +90,30 @@ database. They are never exposed to workspaces, developers, or the browser
|
||||
after initial entry. The dashboard shows only whether a key is set, not the
|
||||
key itself.
|
||||
|
||||
When a provider uses ambient credentials, Coder resolves them from the server
|
||||
environment at request time instead of storing a secret in the database.
|
||||
|
||||
Because the agent loop runs in the control plane, workspaces never need direct
|
||||
access to LLM providers. See
|
||||
[Architecture](./architecture.md#no-api-keys-in-workspaces) for details
|
||||
on this security model.
|
||||
|
||||
## Key policy
|
||||
## Credential selection
|
||||
|
||||
Each provider has three policy flags that control how provider credentials are
|
||||
sourced:
|
||||
Coder Agents use the AI providers configured by administrators. Provider API
|
||||
keys entered by administrators are centralized credentials for the deployment.
|
||||
|
||||
| Setting | Default | Description |
|
||||
|-------------------------|---------|--------------------------------------------------------------------------------------------------------------------------|
|
||||
| Central API key | On | The provider uses deployment-managed credentials configured by an administrator. For most providers, this is an API key. |
|
||||
| Allow user API keys | Off | Developers may supply their own API key for this provider. |
|
||||
| Central key as fallback | Off | When user keys are allowed, fall back to deployment-managed credentials if a developer has not set a personal key. |
|
||||
BYOK for Coder Agents is controlled by the
|
||||
[global AI Gateway BYOK setting](../ai-gateway/auth.md#bring-your-own-key-byok),
|
||||
not by per-provider key policy flags. When BYOK is enabled, users can save a
|
||||
personal API key for any enabled AI provider. When BYOK is disabled, saved user
|
||||
keys are ignored and users cannot add or update personal keys.
|
||||
|
||||
At least one credential source must be enabled. These settings appear in the
|
||||
provider configuration form under **Key policy**.
|
||||
For each provider request, Coder selects credentials in this order:
|
||||
|
||||
The interaction between these flags determines whether a provider is available
|
||||
to a given developer:
|
||||
|
||||
| Central key | User keys allowed | Fallback | Developer has key | Result |
|
||||
|-------------|-------------------|----------|-------------------|----------------------|
|
||||
| On | Off | — | — | Uses central key |
|
||||
| Off | On | — | Yes | Uses developer's key |
|
||||
| Off | On | — | No | Unavailable |
|
||||
| On | On | Off | Yes | Uses developer's key |
|
||||
| On | On | Off | No | Unavailable |
|
||||
| On | On | On | Yes | Uses developer's key |
|
||||
| On | On | On | No | Uses central key |
|
||||
|
||||
When a developer's personal key is present, it always takes precedence over
|
||||
deployment-managed credentials. When user keys are required and fallback is
|
||||
disabled, the provider is unavailable to developers who have not saved a
|
||||
personal key, even if deployment-managed credentials exist. This is
|
||||
intentional: it enforces that each developer authenticates with their own
|
||||
credentials.
|
||||
1. If BYOK is enabled and the user has saved a personal key for the selected
|
||||
provider, Coder uses the user's key.
|
||||
1. Otherwise, Coder uses centralized provider credentials when they are
|
||||
configured.
|
||||
1. If neither a usable user key nor centralized credentials are available, the
|
||||
provider is unavailable for that user.
|
||||
|
||||
## Models
|
||||
|
||||
@@ -131,11 +124,11 @@ generation parameters, and provider-specific options.
|
||||
|
||||
1. Open **Settings** > **Manage Agents** and select the **Models** tab.
|
||||
1. Click **Add** and select the provider for the new model.
|
||||
1. Enter the **Model Identifier** — the exact model string your provider
|
||||
1. Enter the **Model Identifier**, the exact model string your provider
|
||||
expects (e.g., `claude-opus-4-6`, `gpt-5.3-codex`).
|
||||
1. Set a **Display Name** so developers see a human-readable label in the model
|
||||
selector.
|
||||
1. Set the **Context Limit** — the maximum number of tokens in the model's
|
||||
1. Set the **Context Limit**, the maximum number of tokens in the model's
|
||||
context window (e.g., `200000` for Claude Sonnet).
|
||||
1. Configure any provider-specific options (see below).
|
||||
1. Click **Save**.
|
||||
@@ -171,7 +164,7 @@ These options apply to all providers:
|
||||
| Model Identifier | The API model string sent to the provider (e.g., `claude-opus-4-6`). |
|
||||
| Display Name | The label shown to developers in the model selector. |
|
||||
| Context Limit | Maximum tokens in the context window. Used to determine when context compaction triggers. |
|
||||
| Compression Threshold | Percentage (0–100) of context usage at which the agent compresses older messages into a summary. |
|
||||
| Compression Threshold | Percentage (0-100) of context usage at which the agent compresses older messages into a summary. |
|
||||
| Max Output Tokens | Maximum tokens generated per model response. |
|
||||
| Temperature | Controls randomness. Lower values produce more deterministic output. |
|
||||
| Top P | Nucleus sampling threshold. |
|
||||
@@ -238,9 +231,9 @@ are active.
|
||||
|
||||
The model selector uses the following precedence to pre-select a model:
|
||||
|
||||
1. **Last used model** — stored in the browser's local storage.
|
||||
1. **Admin-designated default** — the model marked with the star icon.
|
||||
1. **First available model** — if no default is set and no history exists.
|
||||
1. **Last used model**, stored in the browser's local storage.
|
||||
1. **Admin-designated default**, the model marked with the star icon.
|
||||
1. **First available model**, if no default is set and no history exists.
|
||||
|
||||
Developers cannot add their own providers or models. If no models are
|
||||
configured, the chat interface displays a message directing developers to
|
||||
@@ -284,57 +277,44 @@ and resolution falls through to the next.
|
||||
|
||||
## User API keys (BYOK)
|
||||
|
||||
When an administrator enables **Allow user API keys** on a provider,
|
||||
developers can supply their own API key from the Agents settings page.
|
||||
When [AI Gateway BYOK](../ai-gateway/auth.md#bring-your-own-key-byok) is
|
||||
enabled, developers can supply personal API keys for any enabled AI provider
|
||||
from the Agents settings page.
|
||||
|
||||
### Managing personal API keys
|
||||
|
||||
1. Navigate to the **Agents** page in the Coder dashboard.
|
||||
1. Open **Settings** and select the **API Keys** tab.
|
||||
1. Each provider that allows user keys is listed with a status indicator:
|
||||
- **Key saved** — your personal key is active and will be used for requests.
|
||||
- **Using shared key** — no personal key set, but the central deployment
|
||||
key is available as a fallback.
|
||||
- **No key** — you must add a personal key before you can use this provider.
|
||||
1. Each enabled provider is listed with a status indicator:
|
||||
- **Key saved**, your personal key is active and will be used for requests to
|
||||
that provider.
|
||||
- **Using shared key**, no personal key is set and Coder is using
|
||||
deployment-managed credentials for that provider.
|
||||
- **No key**, no personal key or deployment-managed credential is available.
|
||||
Add a personal key before you use models from this provider.
|
||||
1. Enter your API key and click **Save**.
|
||||
|
||||
Personal API keys are encrypted at rest using the same database encryption
|
||||
used for deployment-managed provider secrets. The dashboard never displays a
|
||||
saved key, only whether one is set.
|
||||
|
||||
### How key selection works
|
||||
|
||||
When you start a chat, the control plane resolves which credential source to
|
||||
use for each provider:
|
||||
|
||||
1. If you have a personal key for the provider, it is used.
|
||||
1. If you do not have a personal key and central key fallback is enabled,
|
||||
deployment-managed credentials are used.
|
||||
1. If you do not have a personal key and fallback is disabled, the provider
|
||||
is unavailable to you. Models from that provider will not appear in the
|
||||
model selector.
|
||||
|
||||
### Removing a personal key
|
||||
|
||||
Click **Remove** on the provider card in the API Keys settings tab. If
|
||||
central key fallback is enabled, subsequent requests will use the shared
|
||||
deployment-managed credentials. If fallback is disabled, the provider becomes
|
||||
unavailable until you add a new personal key.
|
||||
Click **Remove** on the provider card in the API Keys settings tab. Subsequent
|
||||
requests use deployment-managed credentials when they are configured for that
|
||||
provider. If no deployment-managed credential is available, add a new personal
|
||||
key before you use models from that provider.
|
||||
|
||||
## Using an LLM proxy
|
||||
|
||||
Organizations that route LLM traffic through a centralized proxy — such as
|
||||
Coder's AI Gateway or third parties like LiteLLM — can point any provider's **Base URL** at their proxy endpoint.
|
||||
Organizations that route LLM traffic through a centralized proxy, such as
|
||||
LiteLLM or an internal gateway, can point a provider's **Endpoint** or **Base
|
||||
URL** at that upstream proxy endpoint. Enter the API key your proxy expects.
|
||||
|
||||
For example, to route all OpenAI traffic through Coder's AI Gateway:
|
||||
|
||||
1. Add or edit the **OpenAI** provider.
|
||||
1. Set the **Base URL** to your AI Gateway endpoint
|
||||
(e.g., `https://example.coder.com/api/v2/aibridge/openai/v1`).
|
||||
1. Enter the API key your proxy expects.
|
||||
|
||||
Alternatively, use the **OpenAI Compatible** provider type if your proxy serves
|
||||
multiple model families through a single OpenAI-compatible endpoint.
|
||||
Use the **OpenAI Compatible** provider type if your proxy serves multiple model
|
||||
families through a single OpenAI-compatible endpoint. Include the proxy
|
||||
provider's documented OpenAI-compatible path prefix, such as `/v1`, when
|
||||
required.
|
||||
|
||||
This lets you keep existing proxy-level features like per-user budgets, rate
|
||||
limiting, and audit logging while using Coder Agents as the developer interface.
|
||||
|
||||
@@ -143,10 +143,8 @@ auth header for the configured `auth_type`:
|
||||
| `X-Coder-Subchat-Id` | Subchat ID. Only present when the request originates from a child chat. |
|
||||
| `X-Coder-Workspace-Id` | Workspace associated with the chat, if any. |
|
||||
|
||||
These are the same headers Coder sends to LLM providers (see
|
||||
[Coder agents headers](../../ai-gateway/clients/coder-agents.md)) so a
|
||||
first-party MCP server can correlate a tool call back to the
|
||||
originating chat.
|
||||
Coder sends the same identity headers to LLM providers, so a first-party
|
||||
MCP server can correlate a tool call back to the originating chat.
|
||||
|
||||
Because the headers leak chat identity, the option is **off by
|
||||
default** and should only be enabled for first-party or trusted
|
||||
|
||||
@@ -68,10 +68,11 @@ With Tasks, LLM credentials are injected into the workspace as environment
|
||||
variables (e.g. `ANTHROPIC_API_KEY`). With Coder Agents, credentials are
|
||||
configured once in the control plane:
|
||||
|
||||
1. Navigate to the **Agents** page in the Coder dashboard.
|
||||
1. Open **Settings** > **Manage Agents** > **Providers**, pick a provider,
|
||||
enter your API key, and save.
|
||||
1. Under **Models**, add at least one model and set it as the default.
|
||||
1. Navigate to **Admin settings** > **AI** and select **Providers**.
|
||||
1. Add or update a provider with its credentials and upstream endpoint, then
|
||||
save it.
|
||||
1. Navigate to the **Agents** page, open **Settings** > **Manage Agents** >
|
||||
**Models**, add at least one model, and set it as the default.
|
||||
|
||||
You no longer pass API keys in template variables or workspace environment. See https://coder.com/docs/ai-coder/agents/getting-started for more information.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user