opensource/coder
3
0
Fork 0
mirror of https://github.com/coder/coder.git synced 2026-09-21 20:51:01 +08:00
Code Issues Packages Projects Releases Wiki Activity

feat: configure multiple AI Bridge providers of the same type (#23948)

Browse Source 
_Disclaimer: produced mostly by Claude Opus 4.6 following detailed
planning._

## Summary
- Support multiple instances of the same AI Bridge provider type via
indexed env vars (`CODER_AIBRIDGE_PROVIDER_<N>_<KEY>`), following the
`CODER_EXTERNAL_AUTH_<N>_<KEY>` pattern
- Existing single-provider env vars (`CODER_AIBRIDGE_OPENAI_KEY`, etc.)
continue to work unchanged
- Setting both a legacy env var and an indexed provider with the same
name errors at startup to prevent silent misconfiguration
- Mark legacy provider fields (`OpenAI`, `Anthropic`, `Bedrock`) as
deprecated in `AIBridgeConfig` in favor of `Providers`
  ## Example
```sh
CODER_AIBRIDGE_PROVIDER_0_TYPE=anthropic
CODER_AIBRIDGE_PROVIDER_0_NAME=anthropic-corp
CODER_AIBRIDGE_PROVIDER_0_KEY=sk-ant-corp-xxx

CODER_AIBRIDGE_PROVIDER_0_BASE_URL=https://llm-proxy.internal.example.com/anthropic

CODER_AIBRIDGE_PROVIDER_1_TYPE=anthropic
CODER_AIBRIDGE_PROVIDER_1_NAME=anthropic-direct
  CODER_AIBRIDGE_PROVIDER_1_KEY=sk-ant-direct-yyy         
  ```
  Each instance is routed by name:
- /api/v2/aibridge/**anthropic-corp**/v1/messages
- /api/v2/aibridge/**anthropic-direct**/v1/messages
Closes
[AIGOV-157](https://linear.app/codercom/issue/AIGOV-157/spike-to-understand-if-there-is-a-simple-way-to-handle-multi-api-key)

---------

Signed-off-by: Danny Kopping <danny@coder.com>
This commit is contained in:
Danny Kopping
2026-04-15 07:59:37 +00:00
committed by GitHub
parent 730edba87a
commit 08045c2aac
17 changed files with 1326 additions and 163 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ai-coder/ai-gateway/clients/copilot.md
+27
View File
@@ -12,6 +12,33 @@ For general information about GitHub Copilot, see the [GitHub Copilot documentat
For general client configuration requirements, see [AI Gateway Proxy Client Configuration](../ai-gateway-proxy/setup.md#client-configuration).
The sections below cover Copilot-specific setup for each client.
## Provider configuration
Configure a `copilot` provider using the
[indexed provider format](../setup.md#multiple-instances-of-the-same-provider).
Copilot providers use OAuth app installations for authentication rather than
static API keys.
```sh
# GitHub Copilot (Individual)
export CODER_AIBRIDGE_PROVIDER_0_TYPE=copilot
export CODER_AIBRIDGE_PROVIDER_0_NAME=copilot
# GitHub Copilot Business
export CODER_AIBRIDGE_PROVIDER_1_TYPE=copilot
export CODER_AIBRIDGE_PROVIDER_1_NAME=copilot-business
export CODER_AIBRIDGE_PROVIDER_1_BASE_URL=https://api.business.githubcopilot.com
# GitHub Copilot Enterprise
export CODER_AIBRIDGE_PROVIDER_2_TYPE=copilot
export CODER_AIBRIDGE_PROVIDER_2_NAME=copilot-enterprise
export CODER_AIBRIDGE_PROVIDER_2_BASE_URL=https://api.enterprise.githubcopilot.com
```
The default base URL targets the individual Copilot API
(`api.individual.githubcopilot.com`). Override `BASE_URL` for Business or
Enterprise tiers as shown above.
## Copilot CLI
For installation instructions, see [GitHub Copilot CLI documentation](https://docs.github.com/en/copilot/how-tos/copilot-cli/install-copilot-cli).
docs/ai-coder/ai-gateway/setup.md
+89 -2
View File
@@ -92,15 +92,102 @@ proxy between AI Gateway and AWS Bedrock.
coder server
```
### Additional providers and Model Proxies
### GitHub Copilot
AI Gateway can relay traffic to other OpenAI- or Anthropic-compatible services or model proxies like LiteLLM by pointing the base URL variables above at the provider you operate. Share feedback or follow along in the [`aibridge`](https://github.com/coder/aibridge) issue tracker as we expand support for additional providers.
Configure a `copilot` provider using the
[indexed provider format](#multiple-instances-of-the-same-provider) with
separate instances for Individual, Business, and Enterprise tiers.
See [GitHub Copilot — Provider configuration](./clients/copilot.md#provider-configuration)
for full setup instructions.
### ChatGPT
Configure a ChatGPT provider by creating an `openai`-typed instance with the
ChatGPT Codex base URL:
```sh
export CODER_AIBRIDGE_PROVIDER_0_TYPE=openai
export CODER_AIBRIDGE_PROVIDER_0_NAME=chatgpt
export CODER_AIBRIDGE_PROVIDER_0_BASE_URL=https://chatgpt.com/backend-api/codex
```
</div>
> [!NOTE]
> See the [Supported APIs](./reference.md#supported-apis) section below for precise endpoint coverage and interception behavior.
### Multiple instances of the same provider
You can configure multiple instances of the same provider type — for example, to
route different teams to separate API keys, use different base URLs per region, or
connect to both a direct API and a proxy simultaneously. Use indexed environment
variables following the pattern `CODER_AIBRIDGE_PROVIDER_<N>_<KEY>`:
```sh
# Anthropic routed through a corporate proxy
export CODER_AIBRIDGE_PROVIDER_0_TYPE=anthropic
export CODER_AIBRIDGE_PROVIDER_0_NAME=anthropic-corp
export CODER_AIBRIDGE_PROVIDER_0_KEY=sk-ant-corp-xxx
export CODER_AIBRIDGE_PROVIDER_0_BASE_URL=https://llm-proxy.internal.example.com/anthropic
# Anthropic direct (for teams that need direct access)
export CODER_AIBRIDGE_PROVIDER_1_TYPE=anthropic
export CODER_AIBRIDGE_PROVIDER_1_NAME=anthropic-direct
export CODER_AIBRIDGE_PROVIDER_1_KEY=sk-ant-direct-yyy
# Azure-hosted OpenAI deployment
export CODER_AIBRIDGE_PROVIDER_2_TYPE=openai
export CODER_AIBRIDGE_PROVIDER_2_NAME=azure-openai
export CODER_AIBRIDGE_PROVIDER_2_KEY=azure-key-zzz
export CODER_AIBRIDGE_PROVIDER_2_BASE_URL=https://my-deployment.openai.azure.com/
# Anthropic via AWS Bedrock
export CODER_AIBRIDGE_PROVIDER_3_TYPE=anthropic
export CODER_AIBRIDGE_PROVIDER_3_NAME=anthropic-bedrock
export CODER_AIBRIDGE_PROVIDER_3_BEDROCK_REGION=us-west-2
export CODER_AIBRIDGE_PROVIDER_3_BEDROCK_ACCESS_KEY=AKIAIOSFODNN7EXAMPLE
export CODER_AIBRIDGE_PROVIDER_3_BEDROCK_ACCESS_KEY_SECRET=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
coder server
```
Each provider instance gets a unique route based on its `NAME`. Clients send
requests to `/api/v2/aibridge/<NAME>/` to target a specific instance:
| Instance name | Route |
|---------------------|-----------------------------------------------------|
| `anthropic-corp` | `/api/v2/aibridge/anthropic-corp/v1/messages` |
| `anthropic-direct` | `/api/v2/aibridge/anthropic-direct/v1/messages` |
| `azure-openai` | `/api/v2/aibridge/azure-openai/v1/chat/completions` |
| `anthropic-bedrock` | `/api/v2/aibridge/anthropic-bedrock/v1/messages` |
**Supported keys per provider:**
| Key | Required | Description |
|------------|----------|------------------------------------------------------|
| `TYPE` | Yes | Provider type: `openai`, `anthropic`, or `copilot` |
| `NAME` | No | Unique instance name for routing. Defaults to `TYPE` |
| `KEY` | No | API key for upstream authentication (alias: `KEYS`) |
| `BASE_URL` | No | Base URL of the upstream API |
For `anthropic` providers using AWS Bedrock, the following keys are also
available: `BEDROCK_BASE_URL`, `BEDROCK_REGION`,
`BEDROCK_ACCESS_KEY` (alias: `BEDROCK_ACCESS_KEYS`),
`BEDROCK_ACCESS_KEY_SECRET` (alias: `BEDROCK_ACCESS_KEY_SECRETS`),
`BEDROCK_MODEL`, `BEDROCK_SMALL_FAST_MODEL`.
> [!NOTE]
> Indices must be contiguous and start at `0`. Each instance must have a unique
> `NAME` — if two instances of the same `TYPE` omit `NAME`, they will both
> default to the type name and fail with a duplicate name error.
>
> The legacy single-provider environment variables (`CODER_AIBRIDGE_OPENAI_KEY`,
> `CODER_AIBRIDGE_ANTHROPIC_KEY`, etc.) continue to work. However, setting both
> a legacy variable and an indexed provider with the same default name (e.g.
> `CODER_AIBRIDGE_OPENAI_KEY` and an indexed provider named `openai`) will
> produce a startup error — remove one or the other to resolve the conflict.
## Data Retention
AI Gateway records prompts, token usage, tool invocations, and model reasoning for auditing and
docs/reference/api/general.md
Generated
+10
View File
@@ -203,6 +203,16 @@ curl -X GET http://coder-server:8080/api/v2/deployment/config \
"base_url": "string",
"key": "string"
},
"providers": [
{
"base_url": "string",
"bedrock_model": "string",
"bedrock_region": "string",
"bedrock_small_fast_model": "string",
"name": "string",
"type": "string"
}
],
"rate_limit": 0,
"retention": 0,
"send_actor_headers": true,
docs/reference/api/schemas.md
Generated
+82 -17
View File
@@ -455,6 +455,16 @@
"base_url": "string",
"key": "string"
},
"providers": [
{
"base_url": "string",
"bedrock_model": "string",
"bedrock_region": "string",
"bedrock_small_fast_model": "string",
"name": "string",
"type": "string"
}
],
"rate_limit": 0,
"retention": 0,
"send_actor_headers": true,
@@ -464,23 +474,24 @@
### Properties
| Name | Type | Required | Restrictions | Description |
|-------------------------------------|----------------------------------------------------------------------|----------|--------------|-----------------------------------------------------------------------------------------------------------------------|
| `anthropic` | [codersdk.AIBridgeAnthropicConfig](#codersdkaibridgeanthropicconfig) | false | | |
| `bedrock` | [codersdk.AIBridgeBedrockConfig](#codersdkaibridgebedrockconfig) | false | | |
| `circuit_breaker_enabled` | boolean | false | | Circuit breaker protects against cascading failures from upstream AI provider rate limits (429, 503, 529 overloaded). |
| `circuit_breaker_failure_threshold` | integer | false | | |
| `circuit_breaker_interval` | integer | false | | |
| `circuit_breaker_max_requests` | integer | false | | |
| `circuit_breaker_timeout` | integer | false | | |
| `enabled` | boolean | false | | |
| `inject_coder_mcp_tools` | boolean | false | | Deprecated: Injected MCP in AI Bridge is deprecated and will be removed in a future release. |
| `max_concurrency` | integer | false | | |
| `openai` | [codersdk.AIBridgeOpenAIConfig](#codersdkaibridgeopenaiconfig) | false | | |
| `rate_limit` | integer | false | | |
| `retention` | integer | false | | |
| `send_actor_headers` | boolean | false | | |
| `structured_logging` | boolean | false | | |
| Name | Type | Required | Restrictions | Description |
|-------------------------------------|-----------------------------------------------------------------------------|----------|--------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `anthropic` | [codersdk.AIBridgeAnthropicConfig](#codersdkaibridgeanthropicconfig) | false | | Deprecated: Use Providers with indexed CODER_AIBRIDGE_PROVIDER_<N>_* env vars instead. |
| `bedrock` | [codersdk.AIBridgeBedrockConfig](#codersdkaibridgebedrockconfig) | false | | Deprecated: Use Providers with indexed CODER_AIBRIDGE_PROVIDER_<N>_* env vars instead. |
| `circuit_breaker_enabled` | boolean | false | | Circuit breaker protects against cascading failures from upstream AI provider rate limits (429, 503, 529 overloaded). |
| `circuit_breaker_failure_threshold` | integer | false | | |
| `circuit_breaker_interval` | integer | false | | |
| `circuit_breaker_max_requests` | integer | false | | |
| `circuit_breaker_timeout` | integer | false | | |
| `enabled` | boolean | false | | |
| `inject_coder_mcp_tools` | boolean | false | | Deprecated: Injected MCP in AI Bridge is deprecated and will be removed in a future release. |
| `max_concurrency` | integer | false | | |
| `openai` | [codersdk.AIBridgeOpenAIConfig](#codersdkaibridgeopenaiconfig) | false | | Deprecated: Use Providers with indexed CODER_AIBRIDGE_PROVIDER_<N>_* env vars instead. |
| `providers` | array of [codersdk.AIBridgeProviderConfig](#codersdkaibridgeproviderconfig) | false | | Providers holds provider instances populated from CODER_AIBRIDGE_PROVIDER_<N>_<KEY> env vars and/or the deprecated LegacyOpenAI/LegacyAnthropic/LegacyBedrock fields above. |
| `rate_limit` | integer | false | | |
| `retention` | integer | false | | |
| `send_actor_headers` | boolean | false | | |
| `structured_logging` | boolean | false | | |
## codersdk.AIBridgeInterception
@@ -732,6 +743,30 @@
| `base_url` | string | false | | |
| `key` | string | false | | |
## codersdk.AIBridgeProviderConfig
```json
{
"base_url": "string",
"bedrock_model": "string",
"bedrock_region": "string",
"bedrock_small_fast_model": "string",
"name": "string",
"type": "string"
}
```
### Properties
| Name | Type | Required | Restrictions | Description |
|----------------------------|--------|----------|--------------|--------------------------------------------------------------------------------------------|
| `base_url` | string | false | | Base URL is the base URL of the upstream provider API. |
| `bedrock_model` | string | false | | |
| `bedrock_region` | string | false | | |
| `bedrock_small_fast_model` | string | false | | |
| `name` | string | false | | Name is the unique instance identifier used for routing. Defaults to Type if not provided. |
| `type` | string | false | | Type is the provider type: "openai", "anthropic", or "copilot". |
## codersdk.AIBridgeProxyConfig
```json
@@ -1234,6 +1269,16 @@
"base_url": "string",
"key": "string"
},
"providers": [
{
"base_url": "string",
"bedrock_model": "string",
"bedrock_region": "string",
"bedrock_small_fast_model": "string",
"name": "string",
"type": "string"
}
],
"rate_limit": 0,
"retention": 0,
"send_actor_headers": true,
@@ -3258,6 +3303,16 @@ CreateWorkspaceRequest provides options for creating a new workspace. Only one o
"base_url": "string",
"key": "string"
},
"providers": [
{
"base_url": "string",
"bedrock_model": "string",
"bedrock_region": "string",
"bedrock_small_fast_model": "string",
"name": "string",
"type": "string"
}
],
"rate_limit": 0,
"retention": 0,
"send_actor_headers": true,
@@ -3837,6 +3892,16 @@ CreateWorkspaceRequest provides options for creating a new workspace. Only one o
"base_url": "string",
"key": "string"
},
"providers": [
{
"base_url": "string",
"bedrock_model": "string",
"bedrock_region": "string",
"bedrock_small_fast_model": "string",
"name": "string",
"type": "string"
}
],
"rate_limit": 0,
"retention": 0,
"send_actor_headers": true,