opensource/coder
3
0
Fork 0
mirror of https://github.com/coder/coder.git synced 2026-10-29 17:44:33 +08:00
Code Issues Packages Projects Releases Wiki Activity

docs: rename AI Bridge to AI Gateway and Agent Boundaries to Agent Firewall (#24094)

Browse Source 
*Disclaimer: implemented by a Coder Agent using Claude Opus 4.6*

## Summary

Renames product references across documentation:

| Old Name | New Name |
|----------|----------|
| AI Bridge | AI Gateway |
| AI Bridge Proxy | AI Gateway Proxy |
| Agent Boundaries | Agent Firewall |

## What changed

- Prose text, headings, titles, and descriptions updated across all docs
- Directories renamed:
  - `docs/ai-coder/ai-bridge/` → `docs/ai-coder/ai-gateway/`
- `docs/ai-coder/ai-bridge/ai-bridge-proxy/` →
`docs/ai-coder/ai-gateway/ai-gateway-proxy/`
  - `docs/ai-coder/agent-boundaries/` → `docs/ai-coder/agent-firewall/`
- All internal markdown links updated to new paths
- `manifest.json` route paths updated
- Rename notice added to AI Gateway and Agent Firewall entrypoint pages

## Companion PR

URL redirects (old paths → new paths):
[coder/coder.com#700](https://github.com/coder/coder.com/pull/700)

## What is intentionally NOT changed

- **Env vars**: `CODER_AIBRIDGE_*`
- **CLI flags**: `--aibridge-*`
- **API paths**: `/api/v2/aibridge/*`
- **Config keys**: `aibridge:` YAML blocks
- **Terraform variables**: `enable_aibridge`, `boundary_version`,
`use_boundary_directly`
- **Process names**: `aibridged`, `aibridgeproxyd`
- **Prometheus metrics**: `coder_aibridged_*`, `coder_aibridgeproxyd_*`
- **SDK types**: `codersdk.AIBridge*`
- **GitHub URLs**: `github.com/coder/aibridge`
- **Image paths**: `images/aibridge/`
- **Auto-generated reference docs**: `docs/reference/cli/aibridge*.md`,
`docs/reference/api/aibridge.md`, `docs/reference/api/schemas.md`
- **Frontend code**: `site/src/` references (separate PR)

Code-level renames (env vars, configs, frontend) are planned for a
follow-up PR.
This commit is contained in:
Danny Kopping
2026-04-09 10:07:50 +00:00
committed by GitHub
parent 2e6fdf2344
commit 7a94a683c4
39 changed files with 340 additions and 330 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ai-coder/ai-gateway/ai-gateway-proxy/index.md
+35
View File
@@ -0,0 +1,35 @@
# AI Gateway Proxy
AI Gateway Proxy extends [AI Gateway](../index.md) to support clients that don't allow base URL overrides.
While AI Gateway requires clients to support custom base URLs, many popular AI coding tools lack this capability.
AI Gateway Proxy solves this by acting as an HTTP proxy that intercepts traffic to supported AI providers and forwards it to AI Gateway. Since most clients respect proxy configurations even when they don't support base URL overrides, this provides a universal compatibility layer for AI Gateway.
For a list of clients supported through AI Gateway Proxy, see [Client Configuration](../clients/index.md).
## How it works
AI Gateway Proxy operates in two modes depending on the destination:
* MITM (Man-in-the-Middle) mode for allowlisted AI provider domains:
* Intercepts and decrypts HTTPS traffic using a configured CA certificate
* Forwards requests to AI Gateway for authentication, auditing, and routing
* Supports: Anthropic, OpenAI, GitHub Copilot
* Tunnel mode for all other traffic:
* Passes requests through without decryption
Clients authenticate by passing their Coder token in the proxy credentials.
<!-- TODO(ssncferreira): Add diagram showing how AI Gateway Proxy works in tunnel and MITM modes -->
## When to use AI Gateway Proxy
Use AI Gateway Proxy when your AI tools don't support base URL overrides but do respect standard proxy configurations.
For clients that support base URL configuration, you can use [AI Gateway](../index.md) directly.
Nevertheless, clients with base URL overrides also work with the proxy, in case you want to use multiple AI clients and some of them do not support base URL configuration.
## Next steps
* [Set up AI Gateway Proxy](./setup.md) on your Coder deployment
docs/ai-coder/ai-gateway/ai-gateway-proxy/setup.md
+361
View File
@@ -0,0 +1,361 @@
# Setup
AI Gateway Proxy runs inside the Coder control plane (`coderd`), requiring no separate compute to deploy or scale.
Once enabled, `coderd` runs the `aibridgeproxyd` in-memory and intercepts traffic to supported AI providers, forwarding it to AI Gateway.
**Required:**
1. AI Gateway must be enabled and configured (requires a **Premium** license with the [AI Governance Add-On](../../ai-governance.md)). See [AI Gateway Setup](../setup.md) for further information.
1. AI Gateway Proxy must be [enabled](#proxy-configuration) using the server flag.
1. A [CA certificate](#ca-certificate) must be configured for MITM interception.
1. [Clients](#client-configuration) must be configured to use the proxy and trust the CA certificate.
## Proxy Configuration
AI Gateway Proxy is disabled by default. To enable it, set the following configuration options:
```shell
CODER_AIBRIDGE_ENABLED=true \
CODER_AIBRIDGE_PROXY_ENABLED=true \
CODER_AIBRIDGE_PROXY_CERT_FILE=/path/to/ca.crt \
CODER_AIBRIDGE_PROXY_KEY_FILE=/path/to/ca.key \
coder server
# or via CLI flags:
coder server \
--aibridge-enabled=true \
--aibridge-proxy-enabled=true \
--aibridge-proxy-cert-file=/path/to/ca.crt \
--aibridge-proxy-key-file=/path/to/ca.key
```
Both the certificate and private key are required for AI Gateway Proxy to start.
See [CA Certificate](#ca-certificate) for how to generate and obtain these files.
By default, the proxy listener accepts plain HTTP connections.
To serve the listener over HTTPS, provide a TLS certificate and key:
```shell
CODER_AIBRIDGE_PROXY_TLS_CERT_FILE=/path/to/listener.crt
CODER_AIBRIDGE_PROXY_TLS_KEY_FILE=/path/to/listener.key
# or via CLI flags:
--aibridge-proxy-tls-cert-file=/path/to/listener.crt
--aibridge-proxy-tls-key-file=/path/to/listener.key
```
Both files must be provided together.
The TLS certificate must include a Subject Alternative Name (SAN) matching the hostname or IP address that clients use to connect to the proxy.
See [Proxy TLS Configuration](#proxy-tls-configuration) for how to generate and configure these files.
The AI Gateway Proxy only intercepts and forwards traffic to AI Gateway for the supported AI provider domains:
* [Anthropic](https://www.anthropic.com/): `api.anthropic.com`
* [OpenAI](https://openai.com/): `api.openai.com`
* [GitHub Copilot](https://github.com/copilot): `api.individual.githubcopilot.com`
All other traffic is tunneled through without decryption.
For additional configuration options, see the [Coder server configuration](../../../reference/cli/server.md#options).
## Security Considerations
> [!WARNING]
> The AI Gateway Proxy should only be accessible within a trusted network and **must not** be directly exposed to the public internet.
> Without proper network restrictions, unauthorized users could route traffic through the proxy or intercept credentials.
### Encrypting client connections
By default, AI tools send the Coder session token in the proxy credentials over unencrypted HTTP.
This only applies to the initial connection between the client and the proxy.
Once connected:
* MITM mode: A TLS connection is established between the AI tool and the proxy (using the configured CA certificate), then traffic is forwarded securely to AI Gateway.
* Tunnel mode: A TLS connection is established directly between the AI tool and the destination, passing through the proxy without decryption.
As a best practice, apply one or more of the following to protect credentials during the initial connection:
* TLS listener (recommended): Enable TLS directly on the proxy so clients connect over HTTPS.
See [Proxy TLS Configuration](#proxy-tls-configuration) for configuration steps.
* Internal network only: If the proxy and all clients are on the same trusted network, credentials are not exposed to external attackers.
* TLS-terminating load balancer: Place a TLS-terminating load balancer in front of the proxy that terminates TLS and forwards requests over HTTP.
### Restricting proxy access
Requests to non-allowlisted domains are tunneled through the proxy without restriction.
To prevent unauthorized use, restrict network access to the proxy so that only authorized clients can connect.
## CA Certificate
AI Gateway Proxy uses a CA (Certificate Authority) certificate to perform MITM interception of HTTPS traffic.
When AI tools connect to AI provider domains through the proxy, the proxy presents a certificate signed by this CA.
AI tools must trust this CA certificate, otherwise, the connection will fail.
### Self-signed certificate
Use a self-signed certificate when your organization doesn't have an internal CA, or when you want a dedicated CA specifically for AI Gateway Proxy.
Generate a CA certificate specifically for AI Gateway Proxy:
1) Generate a private key:
```shell
openssl genrsa -out ca.key 4096
chmod 400 ca.key
```
1) Create a self-signed CA certificate (valid for 10 years):
```shell
openssl req -new -x509 -days 3650 \
-key ca.key \
-out ca.crt \
-subj "/CN=AI Gateway Proxy CA"
```
Configure AI Gateway Proxy with both files:
```shell
CODER_AIBRIDGE_PROXY_CERT_FILE=/path/to/ca.crt
CODER_AIBRIDGE_PROXY_KEY_FILE=/path/to/ca.key
```
### Corporate CA certificate
If your organization has an internal CA that clients already trust, you can have it issue an intermediate CA certificate for AI Gateway Proxy.
This simplifies deployment since AI tools that already trust your organization's root CA will automatically trust certificates signed by the intermediate.
Your organization's CA issues a certificate and private key pair for the proxy. Configure the proxy with both files:
```shell
CODER_AIBRIDGE_PROXY_CERT_FILE=/path/to/intermediate-ca.crt
CODER_AIBRIDGE_PROXY_KEY_FILE=/path/to/intermediate-ca.key
```
### Securing the private key
> [!WARNING]
> The CA private key is used to sign certificates for MITM interception.
> Store it securely and restrict access. If compromised, an attacker could intercept traffic from any client that trusts the CA certificate.
Best practices:
* Restrict file permissions so only the Coder process can read the key.
* Use a secrets manager to store the key where possible.
### Distributing the certificate
AI tools need to trust the CA certificate before connecting through the proxy.
For **self-signed certificates**, AI tools must be configured to trust the CA certificate. The certificate (without the private key) is available at:
```shell
https://<coder-url>/api/v2/aibridge/proxy/ca-cert.pem
```
For **corporate CA certificates**, if the systems where AI tools run already trust your organization's root CA, and the intermediate certificate chains correctly to that root, no additional certificate distribution is needed.
Otherwise, AI tools must be configured to trust the intermediate CA certificate from the endpoint above.
How you configure AI tools to trust the certificate depends on the tool and operating system. See [Client Configuration](#client-configuration) for details.
## Proxy TLS Configuration
By default, the AI Gateway Proxy listener accepts plain HTTP connections.
When TLS is enabled, the proxy serves over HTTPS, encrypting the connection between AI tools and the proxy.
The TLS certificate is separate from the [MITM CA certificate](#ca-certificate).
The CA certificate is used to sign dynamically generated certificates during MITM interception.
The TLS certificate identifies the proxy itself, like any standard web server certificate.
The AI Gateway Proxy enforces a minimum TLS version of 1.2.
### Configuration
In addition to the required proxy configuration, set the following to enable TLS on the proxy:
```shell
CODER_AIBRIDGE_PROXY_TLS_CERT_FILE=/path/to/listener.crt
CODER_AIBRIDGE_PROXY_TLS_KEY_FILE=/path/to/listener.key
# or via CLI flags:
--aibridge-proxy-tls-cert-file=/path/to/listener.crt
--aibridge-proxy-tls-key-file=/path/to/listener.key
```
Both files must be provided together. If only one is set, the proxy will fail to start.
### Self-signed certificate
Use a self-signed certificate when your organization doesn't have an internal CA, or when you want a dedicated certificate specifically for the AI Gateway Proxy.
The TLS certificate must include a Subject Alternative Name (SAN) matching the hostname or IP address that clients use to connect to the proxy.
Without a matching SAN, clients will reject the connection.
1) Generate a private key:
```shell
openssl genrsa -out listener.key 4096
chmod 400 listener.key
```
1) Create a self-signed certificate:
```shell
openssl req -new -x509 -days 365 \
-key listener.key \
-out listener.crt \
-subj "/CN=<proxy-host>" \
-addext "subjectAltName=DNS:<proxy-host>,IP:<proxy-ip>"
```
Replace `<proxy-host>` and `<proxy-ip>` with the hostname and IP address that clients use to connect to the proxy.
### Corporate CA certificate
If your organization has an internal CA, have it issue a leaf certificate for the proxy.
The certificate must include a SAN matching the proxy's hostname or IP address.
If clients already trust your organization's root CA, no additional certificate configuration is needed for the TLS connection to the proxy.
### Trusting the TLS certificate
For **self-signed certificates**, AI tools must be configured to trust the TLS certificate.
For **corporate CA certificates**, if the systems where AI tools run already trust your organization's root CA, no additional configuration is needed.
How you configure AI tools to trust the certificate depends on the tool and operating system.
See [Client Configuration](#client-configuration) for details.
## Upstream proxy
If your organization requires all outbound traffic to pass through a corporate proxy, you can configure AI Gateway Proxy to chain requests to an upstream proxy.
> [!NOTE]
> AI Gateway Proxy must be the first proxy in the chain.
> AI tools must be configured to connect directly to AI Gateway Proxy, which then forwards tunneled traffic to the upstream proxy.
### How it works
Tunneled requests (non-allowlisted domains) are forwarded to the upstream proxy configured via [`CODER_AIBRIDGE_PROXY_UPSTREAM`](../../../reference/cli/server.md#--aibridge-proxy-upstream).
MITM'd requests (AI provider domains) are forwarded to AI Gateway, which then communicates with AI providers.
To ensure AI Gateway also routes requests through the upstream proxy, make sure to configure the proxy settings for the Coder server process.
<!-- TODO(ssncferreira): Add diagram showing how AI Gateway Proxy integrates with upstream proxies -->
### Configuration
Configure the upstream proxy URL:
```shell
CODER_AIBRIDGE_PROXY_UPSTREAM=http://<corporate-proxy-url>:8080
```
For HTTPS upstream proxies, if the upstream proxy uses a certificate not trusted by the system, provide the CA certificate:
```shell
CODER_AIBRIDGE_PROXY_UPSTREAM=https://<corporate-proxy-url>:8080
CODER_AIBRIDGE_PROXY_UPSTREAM_CA=/path/to/corporate-ca.crt
```
If the system already trusts the upstream proxy's CA certificate, [`CODER_AIBRIDGE_PROXY_UPSTREAM_CA`](../../../reference/cli/server.md#--aibridge-proxy-upstream-ca) is not required.
<!-- TODO(ssncferreira): Add Client Configuration section -->
<!-- TODO(ssncferreira): Add Troubleshooting section -->
## Client Configuration
To use AI Gateway Proxy, AI tools must be configured to:
1. Route traffic through the proxy
1. Trust the proxy's CA certificate
### Configuring the proxy
The preferred approach is to configure the proxy directly in the AI tool's settings, as this avoids routing unnecessary traffic through the proxy.
Consult the tool's documentation for specific instructions.
Alternatively, most tools support the standard `HTTPS_PROXY` environment variable, though this is not guaranteed for all tools:
```shell
export HTTPS_PROXY="https://coder:${CODER_SESSION_TOKEN}@<proxy-host>:8888"
```
Note: if [TLS is not enabled](#proxy-tls-configuration) on the proxy, replace `https://` with `http://` in the proxy URL.
`HTTPS_PROXY` is used for requests to `https://` URLs, which includes all supported AI provider domains.
> [!NOTE]
> `HTTP_PROXY` is not required since AI providers only use `HTTPS`.
> Leaving it unset avoids routing unnecessary traffic through the proxy.
In order for AI tools that communicate with AI Gateway Proxy to authenticate with Coder via AI Gateway, the Coder session token needs to be passed in the proxy credentials as the password field.
### Trusting the CA certificate
The preferred approach is to configure the CA certificate directly in the AI tool's settings, as this limits the scope of the trusted certificate to that specific application.
Consult the tool's documentation for specific instructions.
> [!NOTE]
> If using a [corporate CA certificate](#corporate-ca-certificate) and the system already trusts your organization's root CA, no additional certificate configuration is required.
Download the certificate:
```shell
curl -o coder-aibridge-proxy-ca.pem \
-H "Coder-Session-Token: ${CODER_SESSION_TOKEN}" \
https://<coder-url>/api/v2/aibridge/proxy/ca-cert.pem
```
Replace `<coder-url>` with your Coder deployment URL.
When [TLS is enabled](#proxy-tls-configuration) on the proxy, AI tools must trust both the [MITM CA certificate](#ca-certificate) and the [TLS certificate](#proxy-tls-configuration).
Combine both certificates into a single PEM file:
```shell
cat coder-aibridge-proxy-ca.pem listener.crt > combined-ca.pem
```
Use this combined file for any of the environment variables listed below.
#### Environment variables
Different AI tools use different runtimes, each with their own environment variable for CA certificates:
| Environment Variable | Runtime |
|-----------------------|---------------------------|
| `NODE_EXTRA_CA_CERTS` | Node.js |
| `SSL_CERT_FILE` | OpenSSL, Python, curl |
| `REQUESTS_CA_BUNDLE` | Python `requests` library |
| `CURL_CA_BUNDLE` | curl |
Set the environment variables associated with the AI tool's runtime.
If you're unsure which runtime the tool uses, or if you use multiple AI tools, the simplest approach is to set all of them:
```shell
export NODE_EXTRA_CA_CERTS="/path/to/coder-aibridge-proxy-ca.pem"
export SSL_CERT_FILE="/path/to/coder-aibridge-proxy-ca.pem"
export REQUESTS_CA_BUNDLE="/path/to/coder-aibridge-proxy-ca.pem"
export CURL_CA_BUNDLE="/path/to/coder-aibridge-proxy-ca.pem"
```
#### System trust store
When tool-specific or environment variable configuration is not possible, you can add the certificate to the system trust store.
This makes the certificate trusted by all applications on the system.
On Linux:
```shell
sudo cp coder-aibridge-proxy-ca.pem /usr/local/share/ca-certificates/
sudo update-ca-certificates
```
For other operating systems, refer to the system's documentation for instructions on adding trusted certificates.
### Coder workspaces
For AI tools running inside Coder workspaces, template administrators can pre-configure the proxy settings and CA certificate in the workspace template.
This provides a seamless experience where users don't need to configure anything manually.
<!-- TODO(ssncferreira): Add registry link for AI Gateway Proxy module for Coder workspaces: https://github.com/coder/internal/issues/1187 -->
For tool-specific configuration details, check the [client compatibility table](../clients/index.md#compatibility) for clients that require proxy-based integration.
docs/ai-coder/ai-gateway/audit.md
+109
View File
@@ -0,0 +1,109 @@
# Auditing AI Sessions
AI Gateway groups intercepted requests into **sessions** and **threads** to show
the causal relationships between human prompts and agent actions. This
structure gives auditors clear provenance over who initiated what, and why.
## Concepts
| Term | Definition |
|------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| **Interception** | A single intercepted request/response pair between client and provider. | |
| **Thread** | A multi-part interaction starting with a human prompt that triggers one or more tool calls, forming an agentic loop. |
| **Agentic loop** | A sequence of tool invocations the agent performs to satisfy a request. The model ends its turn with a tool call, the client invokes it, sends the result back, and the cycle repeats until the model has enough information to formulate a response. |
| **Session** | A set of threads grouped by a client-provided session key. Claude Code and Codex provide session IDs automatically; other clients may not. |
## Human vs. Agent attribution
AI Gateway distinguishes between human-initiated and agent-initiated requests
using the `role` property:
- A message with `role="user"` indicates a human-initiated action (i.e. prompt).
- A message with `role="assistant"` indicates a message generated by a model.
- A message with `role="system"` indicates the system prompt for the client.
The `user` role is currently overloaded by clients like Claude Code and Codex;
they inject system instructions
within `role="user"` blocks when using agents. AI Gateway applies a heuristic
of storing only the **last** prompt from a block of `role="user"` messages.
> [!NOTE]
> AI Gateway cannot declare with certainty whether a request was human- or
> agent-initiated.
## LLM reasoning capture
AI Gateway captures model reasoning and thinking content when available. Both
Anthropic (extended thinking) and OpenAI (reasoning summaries) support this
feature. Reasoning data gives auditors insight into **why** a tool was called,
not just what was called.
## Navigating the UI
### Sessions list
The sessions page (`http://<deployment-url>/aibridge/sessions`) lists all sessions in
reverse-chronological order. Each row shows the last prompt, initiator, provider,
client, token usage, thread count, and timestamp.
Select one to view its full details.
![Sessions](../../images/aibridge/sessions.png)
### Session detail
Click into a session to see a chronological causal chain of events.
Within a thread, each step shows token usage, tool call details (including
arguments and MCP server URLs), duration, and any errors or warnings.
![Session detail](../../images/aibridge/session_detail.png)
## Conducting a forensic audit
When investigating an incident (policy violation, destructive action, etc.):
1. **Identify the session.** Filter by user, time range, or client to find the
relevant session.
1. **Locate the thread.** Each thread in a session shows the (likely) human prompt
that initiated the chain of actions.
1. **Trace the causal chain.** Expand the thread to see every step in the
agentic loop — each tool call and its arguments.
1. **Review model reasoning.** If extended thinking was enabled, check the
model's reasoning at each step to understand why specific tools were called.
1. **Assess attribution.** The session identifies the human who
initiated the action. Subsequent interceptions represent agent-driven actions
that stem from that original prompt.
## What we store
AI Gateway captures the following data from each request/response:
- Last user prompt
- Token usage
- Tool calls (requests only, not responses)
- Responses may be very large, and generally have lower audit value than requests
- In future, we will support storing these results
- Model thinking/reasoning
Model-produced inference text is discarded, as generated text alone
cannot affect external systems. The retention philosophy prioritizes:
- **Human prompts** — capture intent and detect policy violations or
exfiltration attempts.
- **Tool calls** — record how agents interact with external systems,
which is critical for understanding how incidents occurred. For
example, an agent might delete and recreate a database because it
lacks permissions to satisfy a human request to query a table.
- **Model reasoning** — preserve thinking content that explains why
specific tools were invoked, distinguishing between human instruction
and model misunderstanding as the root cause.
See [data retention](./setup.md#data-retention) to configure how long
session data is kept.
## Next steps
- [Monitoring](./monitoring.md) — Dashboards, data export, and tracing
- [Setup](./setup.md) — Configure AI Gateway and data retention
- [Reference](./reference.md) — API and technical reference
docs/ai-coder/ai-gateway/clients/claude-code.md
+92
View File
@@ -0,0 +1,92 @@
# Claude Code
Claude Code can be configured using environment variables. All modes require a **[Coder session token](../../../admin/users/sessions-tokens.md#generate-a-long-lived-api-token-on-behalf-of-yourself)** for authentication with AI Gateway.
## Centralized API Key
```bash
# AI Gateway base URL.
export ANTHROPIC_BASE_URL="<your-deployment-url>/api/v2/aibridge/anthropic"
# Your Coder session token, used for authentication with AI Gateway.
export ANTHROPIC_AUTH_TOKEN="<your-coder-session-token>"
```
## BYOK (Personal API Key)
```bash
# AI Gateway base URL.
export ANTHROPIC_BASE_URL="<your-deployment-url>/api/v2/aibridge/anthropic"
# Your personal Anthropic API key, forwarded to Anthropic.
export ANTHROPIC_API_KEY="<your-anthropic-api-key>"
# Your Coder session token, used for authentication with AI Gateway.
export ANTHROPIC_CUSTOM_HEADERS="X-Coder-AI-Governance-Token: <your-coder-session-token>"
# Ensure no auth token is set so Claude Code uses the API key instead.
unset ANTHROPIC_AUTH_TOKEN
```
## BYOK (Claude Subscription)
```bash
# AI Gateway base URL.
export ANTHROPIC_BASE_URL="<your-deployment-url>/api/v2/aibridge/anthropic"
# Your Coder session token, used for authentication with AI Gateway.
export ANTHROPIC_CUSTOM_HEADERS="X-Coder-AI-Governance-Token: <your-coder-session-token>"
# Ensure no auth token is set so Claude Code uses subscription login instead.
unset ANTHROPIC_AUTH_TOKEN
```
When you run Claude Code, it will prompt you to log in with your Anthropic
account.
## Pre-configuring in Templates
Template admins can pre-configure Claude Code for a seamless experience. Admins can automatically inject the user's Coder session token and the AI Gateway base URL into the workspace environment.
```hcl
module "claude-code" {
source = "registry.coder.com/coder/claude-code/coder"
version = "4.7.3"
agent_id = coder_agent.main.id
workdir = "/path/to/project" # Set to your project directory
enable_aibridge = true
}
```
### Coder Tasks
[Coder Tasks](../../tasks.md) provides a framework for agents to complete background development operations autonomously. Claude Code can be configured in your Tasks automatically:
```hcl
resource "coder_ai_task" "task" {
count = data.coder_workspace.me.start_count
app_id = module.claude-code.task_app_id
}
data "coder_task" "me" {}
module "claude-code" {
source = "registry.coder.com/coder/claude-code/coder"
version = "4.7.3"
agent_id = coder_agent.main.id
workdir = "/path/to/project" # Set to your project directory
ai_prompt = data.coder_task.me.prompt
# Route through AI Gateway (Premium feature)
enable_aibridge = true
}
```
## VS Code Extension
The Claude Code VS Code extension is also supported.
1. If pre-configured in the workspace environment variables (as shown above), it typically respects them.
2. You may need to sign in once; afterwards, it respects the workspace environment variables.
**References:** [Claude Code Settings](https://docs.claude.com/en/docs/claude-code/settings#environment-variables)
docs/ai-coder/ai-gateway/clients/cline.md
+36
View File
@@ -0,0 +1,36 @@
# Cline
Cline supports both OpenAI and Anthropic models and can be configured to use AI Gateway by setting providers.
## Configuration
To configure Cline to use AI Gateway, follow these steps:
![Cline Settings](../../../images/aibridge/clients/cline-setup.png)
<div class="tabs">
### OpenAI Compatible
1. Open Cline in VS Code.
1. Go to **Settings**.
1. **API Provider**: Select **OpenAI Compatible**.
1. **Base URL**: Enter `https://coder.example.com/api/v2/aibridge/openai/v1`.
1. **OpenAI Compatible API Key**: Enter your **[Coder Session Token](../../../admin/users/sessions-tokens.md#generate-a-long-lived-api-token-on-behalf-of-yourself)**.
1. **Model ID** (Optional): Enter the model you wish to use (e.g., `gpt-5.2-codex`).
![Cline OpenAI Settings](../../../images/aibridge/clients/cline-openai.png)
### Anthropic
1. Open Cline in VS Code.
1. Go to **Settings**.
1. **API Provider**: Select **Anthropic**.
1. **Anthropic API Key**: Enter your **Coder Session Token**.
1. **Base URL**: Enter `https://coder.example.com/api/v2/aibridge/anthropic` after checking **_Use custom base URL_**.
1. **Model ID** (Optional): Select your desired Claude model.
![Cline Anthropic Settings](../../../images/aibridge/clients/cline-anthropic.png)
</div>
**References:** [Cline Configuration](https://github.com/cline/cline)
docs/ai-coder/ai-gateway/clients/codex.md
+97
View File
@@ -0,0 +1,97 @@
# Codex CLI
Codex CLI can be configured to use AI Gateway by setting up a custom model provider.
## Centralized API Key
To configure Codex CLI to use AI Gateway, set the following configuration options in your Codex configuration file (e.g., `~/.codex/config.toml`):
```toml
model_provider = "aibridge"
[model_providers.aibridge]
name = "AI Bridge"
base_url = "<your-deployment-url>/api/v2/aibridge/openai/v1"
env_key = "OPENAI_API_KEY"
wire_api = "responses"
```
To authenticate with AI Gateway, get your **[Coder session token](../../../admin/users/sessions-tokens.md#generate-a-long-lived-api-token-on-behalf-of-yourself)** and set it in your environment:
```bash
export OPENAI_API_KEY="<your-coder-session-token>"
```
Run Codex as usual. It will automatically use the `aibridge` model provider from your configuration.
## BYOK (Personal API Key)
Add the following to your Codex configuration file (e.g., `~/.codex/config.toml`):
```toml
model_provider = "aibridge"
[model_providers.aibridge]
name = "AI Bridge"
base_url = "<your-deployment-url>/api/v2/aibridge/openai/v1"
wire_api = "responses"
requires_openai_auth = true
env_http_headers = { "X-Coder-AI-Governance-Token" = "CODER_SESSION_TOKEN" }
```
Set both environment variables:
```bash
# Your personal OpenAI API key, forwarded to OpenAI.
export OPENAI_API_KEY="<your-openai-api-key>"
# Your Coder session token, used for authentication with AI Gateway.
export CODER_SESSION_TOKEN="<your-coder-session-token>"
```
## BYOK (ChatGPT Subscription)
Add the following to your Codex configuration file (e.g., `~/.codex/config.toml`):
```toml
model_provider = "aibridge"
[model_providers.aibridge]
name = "AI Bridge"
base_url = "<your-deployment-url>/api/v2/aibridge/chatgpt/v1"
wire_api = "responses"
requires_openai_auth = true
env_http_headers = { "X-Coder-AI-Governance-Token" = "CODER_SESSION_TOKEN" }
```
> [!NOTE]
> The `base_url` uses `/aibridge/chatgpt/v1` instead of `/aibridge/openai/v1` to route requests through the ChatGPT provider.
Set your Coder session token and ensure `OPENAI_API_KEY` is not set:
```bash
# Your Coder session token, used for authentication with AI Gateway.
export CODER_SESSION_TOKEN="<your-coder-session-token>"
# Ensure no OpenAI API key is set so Codex uses ChatGPT login instead.
unset OPENAI_API_KEY
```
When you run Codex, it will prompt you to log in with your ChatGPT account.
## Pre-configuring in Templates
If configuring within a Coder workspace, you can use the
[Codex CLI](https://registry.coder.com/modules/coder-labs/codex) module:
```tf
module "codex" {
source = "registry.coder.com/coder-labs/codex/coder"
version = "~> 4.1"
agent_id = coder_agent.main.id
workdir = "/path/to/project" # Set to your project directory
enable_aibridge = true
}
```
**References:** [Codex CLI Configuration](https://developers.openai.com/codex/config-advanced)
docs/ai-coder/ai-gateway/clients/copilot.md
+158
View File
@@ -0,0 +1,158 @@
# GitHub Copilot
[GitHub Copilot](https://github.com/features/copilot) is an AI coding assistant that doesn't support custom base URLs but does respect proxy configurations.
This makes it compatible with [AI Gateway Proxy](../ai-gateway-proxy/index.md), which integrates with [AI Gateway](../index.md) for full access to auditing and governance features.
To use Copilot with AI Gateway, make sure AI Gateway Proxy is properly configured, see [AI Gateway Proxy Setup](../ai-gateway-proxy/setup.md) for instructions.
Copilot uses **per-user tokens** tied to GitHub accounts rather than a shared API key.
Users must still authenticate with GitHub to use Copilot.
For general information about GitHub Copilot, see the [GitHub Copilot documentation](https://docs.github.com/en/copilot).
For general client configuration requirements, see [AI Gateway Proxy Client Configuration](../ai-gateway-proxy/setup.md#client-configuration).
The sections below cover Copilot-specific setup for each client.
## Copilot CLI
For installation instructions, see [GitHub Copilot CLI documentation](https://docs.github.com/en/copilot/how-tos/copilot-cli/install-copilot-cli).
### Proxy configuration
Set the `HTTPS_PROXY` environment variable:
```shell
export HTTPS_PROXY="https://coder:${CODER_SESSION_TOKEN}@<proxy-host>:8888"
```
Replace `<proxy-host>` with your AI Gateway Proxy hostname.
Note: if [TLS is not enabled](../ai-gateway-proxy/setup.md#proxy-tls-configuration) on the proxy, replace `https://` with `http://` in the proxy URL.
### CA certificate trust
Copilot CLI is built on Node.js and uses the `NODE_EXTRA_CA_CERTS` environment variable for custom certificates:
```shell
export NODE_EXTRA_CA_CERTS="/path/to/coder-aibridge-proxy-ca.pem"
```
See [Client Configuration CA certificate trust](../ai-gateway-proxy/setup.md#trusting-the-ca-certificate) for details on how to obtain the certificate file.
When [TLS is enabled](../ai-gateway-proxy/setup.md#proxy-tls-configuration) on the proxy, combine the MITM CA certificate and the TLS certificate into a single file:
```shell
cat coder-aibridge-proxy-ca.pem listener.crt > combined-ca.pem
export NODE_EXTRA_CA_CERTS="/path/to/combined-ca.pem"
```
Copilot CLI may start MCP server processes that use runtimes other than Node.js (e.g. Go).
These processes inherit environment variables like `HTTPS_PROXY` but may not respect `NODE_EXTRA_CA_CERTS`.
Adding the TLS certificate to the [system trust store](../ai-gateway-proxy/setup.md#system-trust-store) ensures all processes trust it.
## VS Code Copilot Extension
For installation instructions, see [Installing the GitHub Copilot extension in VS Code](https://docs.github.com/en/copilot/how-tos/set-up/install-copilot-extension?tool=vscode).
### Proxy configuration
You can configure the proxy using environment variables or VS Code settings.
For environment variables, see [AI Gateway Proxy client configuration](../ai-gateway-proxy/setup.md#configuring-the-proxy).
Alternatively, you can configure the proxy directly in VS Code settings:
1. Open Settings (`Ctrl+,` for Windows or `Cmd+,` for macOS)
1. Search for `HTTP: Proxy`
1. Set the proxy URL using the format `https://coder:<CODER_SESSION_TOKEN>@<proxy-host>:8888`
Or add directly to your `settings.json`:
```json
{
"http.proxy": "https://coder:<CODER_SESSION_TOKEN>@<proxy-host>:8888"
}
```
Note: if [TLS is not enabled](../ai-gateway-proxy/setup.md#proxy-tls-configuration) on the proxy, replace `https://` with `http://` in the proxy URL.
The `http.proxy` setting is used for both HTTP and HTTPS requests.
Replace `<proxy-host>` with your AI Gateway Proxy hostname and `<CODER_SESSION_TOKEN>` with your coder session token.
Restart VS Code for changes to take effect.
For more details, see [Configuring proxy settings for Copilot](https://docs.github.com/en/copilot/how-tos/configure-personal-settings/configure-network-settings?tool=vscode) in the GitHub documentation.
### CA certificate trust
Add the AI Gateway Proxy CA certificate to your operating system's trust store.
By default, VS Code loads system certificates, controlled by the `http.systemCertificates` setting.
See [Client Configuration CA certificate trust](../ai-gateway-proxy/setup.md#trusting-the-ca-certificate) for details on how to obtain the certificate file.
When [TLS is enabled](../ai-gateway-proxy/setup.md#proxy-tls-configuration) on the proxy, add the TLS certificate to the system trust store as well.
### Using Coder Remote extension
When connecting to a Coder workspace with the [Coder extension](https://marketplace.visualstudio.com/items?itemName=coder.coder-remote), the Copilot extension runs inside the Coder workspace and not on your local machine.
This means proxy and certificate configuration must be done in the Coder workspace environment.
When [TLS is enabled](../ai-gateway-proxy/setup.md#proxy-tls-configuration) on the proxy, add the TLS certificate to the workspace's system trust store as well.
#### Proxy configuration
Configure the proxy in VS Code's remote settings:
1. [Connect to your Coder workspace](../../../user-guides/workspace-access/vscode.md)
1. Open Settings (`Ctrl+,` for Windows or `Cmd+,` for macOS)
1. Select the **Remote** tab
1. Search for `HTTP: Proxy`
1. Set the proxy URL using the format `https://coder:<CODER_SESSION_TOKEN>@<proxy-host>:8888`
Note: if [TLS is not enabled](../ai-gateway-proxy/setup.md#proxy-tls-configuration) on the proxy, replace `https://` with `http://` in the proxy URL.
Replace `<proxy-host>` with your AI Gateway Proxy hostname and `<CODER_SESSION_TOKEN>` with your coder session token.
#### CA certificate trust
Since the Copilot extension runs inside the Coder workspace, add the [AI Gateway Proxy CA certificate](../ai-gateway-proxy/setup.md#trusting-the-ca-certificate) to the Coder workspace's system trust store.
See [System trust store](../ai-gateway-proxy/setup.md#system-trust-store) for instructions on how to do this on Linux.
Restart VS Code for changes to take effect.
## JetBrains IDEs
For installation instructions, see [Installing the GitHub Copilot extension in JetBrains IDE](https://docs.github.com/en/copilot/how-tos/set-up/install-copilot-extension?tool=jetbrains).
### Proxy configuration
Configure the proxy directly in JetBrains IDE settings:
1. Open Settings (`Ctrl+Alt+S` for Windows or `Cmd+,` for macOS)
1. Navigate to `Appearance & Behavior` > `System Settings` > `HTTP Proxy`
1. Select `Manual proxy configuration` and `HTTP`
1. Enter the proxy hostname and port (default: 8888)
1. Select `Proxy authentication` and enter:
1. Login: `coder` (this value is ignored)
1. Password: Your Coder session token
1. Check `Remember` to save the password
1. Restart the IDE for changes to take effect
For more details, see [Configuring proxy settings for Copilot](https://docs.github.com/en/copilot/how-tos/configure-personal-settings/configure-network-settings?tool=jetbrains) in the GitHub documentation.
### CA certificate trust
Add the AI Gateway Proxy CA certificate to your operating system's trust store.
If the certificate is in the system trust store, no additional IDE configuration is needed.
When [TLS is enabled](../ai-gateway-proxy/setup.md#proxy-tls-configuration) on the proxy, add the TLS certificate to the system trust store as well, or add it under `Accepted certificates` in the IDE settings below.
Alternatively, you can configure the IDE to accept the certificate:
1. Open Settings (`Ctrl+Alt+S` for Windows or `Cmd+,` for macOS)
1. Navigate to `Appearance & Behavior` > `System Settings` > `Server Certificates`
1. Under `Accepted certificates`, click `+` and select the CA certificate file
1. Check `Accept non-trusted certificates automatically`
1. Restart the IDE for changes to take effect
For more details, see [Trusted root certificates](https://www.jetbrains.com/help/idea/ssl-certificates.html) in the JetBrains documentation.
See [Client Configuration CA certificate trust](../ai-gateway-proxy/setup.md#trusting-the-ca-certificate) for details on how to obtain the certificate file.
docs/ai-coder/ai-gateway/clients/factory.md
+35
View File
@@ -0,0 +1,35 @@
# Factory
Factort's Droid agent can be configured to use AI Gateway by setting up custom models for OpenAI and Anthropic.
## Configuration
1. Open `~/.factory/settings.json` (create it if it does not exist).
2. Add a `customModels` entry for each provider you want to use with AI Gateway.
3. Replace `coder.example.com` with your Coder deployment URL.
4. Use a **[Coder session token](../../../admin/users/sessions-tokens.md#generate-a-long-lived-api-token-on-behalf-of-yourself)** for `apiKey`.
```json
{
"customModels": [
{
"model": "claude-4-5-opus",
"displayName": "Claude (Coder AI Bridge)",
"baseUrl": "https://coder.example.com/api/v2/aibridge/anthropic",
"apiKey": "<your-coder-session-token>",
"provider": "anthropic",
"maxOutputTokens": 8192
},
{
"model": "gpt-5.2-codex",
"displayName": "GPT (Coder AI Bridge)",
"baseUrl": "https://coder.example.com/api/v2/aibridge/openai/v1",
"apiKey": "<your-coder-session-token>",
"provider": "openai",
"maxOutputTokens": 16384
}
]
}
```
**References:** [Factory BYOK OpenAI & Anthropic](https://docs.factory.ai/cli/byok/openai-anthropic)
docs/ai-coder/ai-gateway/clients/index.md
+132
View File
@@ -0,0 +1,132 @@
# Client Configuration
Once AI Gateway is setup on your deployment, the AI coding tools used by your users will need to be configured to route requests via AI Gateway.
There are two ways to connect AI tools to AI Gateway:
- Base URL configuration (Recommended): Most AI tools allow customizing the base URL for API requests. This is the preferred approach when supported.
- AI Gateway Proxy: For tools that don't support base URL configuration, [AI Gateway Proxy](../ai-gateway-proxy/index.md) can intercept traffic and forward it to AI Gateway.
## Base URLs
Most AI coding tools allow the "base URL" to be customized. In other words, when a request is made to OpenAI's API from your coding tool, the API endpoint such as [`/v1/chat/completions`](https://platform.openai.com/docs/api-reference/chat) will be appended to the configured base. Therefore, instead of the default base URL of `https://api.openai.com/v1`, you'll need to set it to `https://coder.example.com/api/v2/aibridge/openai/v1`.
The exact configuration method varies by client — some use environment variables, others use configuration files or UI settings:
- **OpenAI-compatible clients**: Set the base URL (commonly via the `OPENAI_BASE_URL` environment variable) to `https://coder.example.com/api/v2/aibridge/openai/v1`
- **Anthropic-compatible clients**: Set the base URL (commonly via the `ANTHROPIC_BASE_URL` environment variable) to `https://coder.example.com/api/v2/aibridge/anthropic`
Replace `coder.example.com` with your actual Coder deployment URL.
## Authentication
Instead of distributing provider-specific API keys (OpenAI/Anthropic keys) to users, they authenticate to AI Gateway using their **Coder session token** or **API key**:
- **OpenAI clients**: Users set `OPENAI_API_KEY` to their Coder session token or API key
- **Anthropic clients**: Users set `ANTHROPIC_API_KEY` to their Coder session token or API key
> [!NOTE]
> Only Coder-issued tokens can authenticate users against AI Gateway.
> AI Gateway will use provider-specific API keys to [authenticate against upstream AI services](../setup.md#configure-providers).
Again, the exact environment variable or setting naming may differ from tool to tool. See a list of [supported clients](#all-supported-clients) below and consult your tool's documentation for details.
### Retrieving your session token
If you're logged in with the Coder CLI, you can retrieve your current session
token using [`coder login token`](../../../reference/cli/login_token.md):
```sh
export ANTHROPIC_API_KEY=$(coder login token)
export ANTHROPIC_BASE_URL="https://coder.example.com/api/v2/aibridge/anthropic"
```
Alternatively, [generate a long-lived API token](../../../admin/users/sessions-tokens.md#generate-a-long-lived-api-token-on-behalf-of-yourself) via the Coder dashboard.
## Bring Your Own Key (BYOK)
In addition to centralized key management, AI Gateway supports **Bring Your
Own Key** (BYOK) mode. Users can provide their own LLM API keys or use
provider subscriptions (such as Claude Pro/Max or ChatGPT Plus/Pro) while
AI Gateway continues to provide observability and governance.
![BYOK authentication flow](../../../images/aibridge/clients/byok_auth_flow.png)
In BYOK mode, users need two credentials:
- A **Coder session token** to authenticate with AI Gateway.
- Their **own LLM credential** (personal API key or subscription token) which AI Gateway forwards
to the upstream provider.
BYOK and centralized modes can be used together. When a user provides
their own credential, AI Gateway forwards it directly. When no user
credential is present, AI Gateway falls back to the admin-configured
provider key. This lets organizations offer centralized keys as a default
while allowing individual users to bring their own.
See individual client pages for configuration details.
## Compatibility
The table below shows tested AI clients and their compatibility with AI Gateway.
| Client | OpenAI | Anthropic | Notes |
|----------------------------------|--------|-----------|--------------------------------------------------------------------------------------------------------------------------------------------------------|
| [Mux](./mux.md) | ✅ | ✅ | |
| [Claude Code](./claude-code.md) | - | ✅ | |
| [Codex CLI](./codex.md) | ✅ | - | |
| [OpenCode](./opencode.md) | ✅ | ✅ | |
| [Factory](./factory.md) | ✅ | ✅ | |
| [Cline](./cline.md) | ✅ | ✅ | |
| [Kilo Code](./kilo-code.md) | ✅ | ✅ | |
| [Roo Code](./roo-code.md) | ✅ | ✅ | |
| [VS Code](./vscode.md) | ✅ | ❌ | Only supports Custom Base URL for OpenAI. |
| [JetBrains IDEs](./jetbrains.md) | ✅ | ❌ | Works in Chat mode via "Bring Your Own Key". |
| [Zed](./zed.md) | ✅ | ✅ | |
| [GitHub Copilot](./copilot.md) | ⚙️ | - | Requires [AI Gateway Proxy](../ai-gateway-proxy/index.md). Uses per-user GitHub tokens. |
| WindSurf | ❌ | ❌ | No option to override base URL. |
| Cursor | ❌ | ❌ | Override for OpenAI broken ([upstream issue](https://forum.cursor.com/t/requests-are-sent-to-incorrect-endpoint-when-using-base-url-override/144894)). |
| Sourcegraph Amp | ❌ | ❌ | No option to override base URL. |
| Kiro | ❌ | ❌ | No option to override base URL. |
| Gemini CLI | ❌ | ❌ | No Gemini API support. Upvote [this issue](https://github.com/coder/aibridge/issues/27). |
| Antigravity | ❌ | ❌ | No option to override base URL. |
|
*Legend: ✅ supported, ⚙️ requires AI Gateway Proxy, ❌ not supported, - not applicable.*
## Configuring In-Workspace Tools
AI coding tools running inside a Coder workspace, such as IDE extensions, can be configured to use AI Gateway.
While users can manually configure these tools with a long-lived API key, template admins can provide a more seamless experience by pre-configuring them. Admins can automatically inject the user's session token with `data.coder_workspace_owner.me.session_token` and the AI Gateway base URL into the workspace environment.
In this example, Claude Code respects these environment variables and will route all requests via AI Gateway.
```hcl
data "coder_workspace_owner" "me" {}
data "coder_workspace" "me" {}
resource "coder_agent" "dev" {
arch = "amd64"
os = "linux"
dir = local.repo_dir
env = {
ANTHROPIC_BASE_URL : "${data.coder_workspace.me.access_url}/api/v2/aibridge/anthropic",
ANTHROPIC_AUTH_TOKEN : data.coder_workspace_owner.me.session_token
}
... # other agent configuration
}
```
## External and Desktop Clients
You can also configure AI tools running outside of a Coder workspace, such as local IDE extensions or desktop applications, to connect to AI Gateway.
The configuration is the same: point the tool to the AI Gateway [base URL](#base-urls) and use a Coder API key for authentication.
Users can generate a long-lived API key from the Coder UI or CLI. Follow the instructions at [Sessions and API tokens](../../../admin/users/sessions-tokens.md#generate-a-long-lived-api-token-on-behalf-of-yourself) to create one.
## All Supported Clients
<children></children>
docs/ai-coder/ai-gateway/clients/jetbrains.md
+35
View File
@@ -0,0 +1,35 @@
# JetBrains IDEs
JetBrains IDE (IntelliJ IDEA, PyCharm, WebStorm, etc.) support AI Gateway via the ["Bring Your Own Key" (BYOK)](https://www.jetbrains.com/help/ai-assistant/use-custom-models.html#provide-your-own-api-key) feature.
## Prerequisites
* [**JetBrains AI Assistant**](https://www.jetbrains.com/help/ai-assistant/installation-guide-ai-assistant.html): Installed and enabled.
* **Authentication**: Your **[Coder session token](../../../admin/users/sessions-tokens.md#generate-a-long-lived-api-token-on-behalf-of-yourself)**.
## Configuration
1. **Open Settings**: Go to **Settings** > **Tools** > **AI Assistant** > **Models & API Keys**.
1. **Configure Provider**: Go to **Third-party AI providers**.
1. **Choose Provider**: Choose **OpenAI-compatible**.
1. **URL**: `https://coder.example.com/api/v2/aibridge/openai/v1`
1. **API Key**: Paste your **[Coder session token](../../../admin/users/sessions-tokens.md#generate-a-long-lived-api-token-on-behalf-of-yourself)**.
1. **Apply**: Click **Apply** and **OK**.
![JetBrains AI Assistant Settings](../../../images/aibridge/clients/jetbrains-ai-settings.png)
## Using the AI Assistant
1. Go back to **AI Chat** on theleft side bar and choose **Chat**.
1. In the Model dropdown, select the desired model (e.g., `gpt-5.2`).
![JetBrains AI Assistant Chat](../../../images/aibridge/clients/jetbrains-ai-chat.png)
You can now use the AI Assistant chat with the configured provider.
> [!NOTE]
>
> * JetBrains AI Assistant currently only supports OpenAI-compatible endpoints. There is an open [issue](https://youtrack.jetbrains.com/issue/LLM-22740) tracking support for Anthropic.
> * JetBrains AI Assistant may not support all models that support OPenAI's `/chat/completions` endpoint in Chat mode.
**References:** [Use custom models with JetBrains AI Assistant](https://www.jetbrains.com/help/ai-assistant/use-custom-models.html#provide-your-own-api-key)
docs/ai-coder/ai-gateway/clients/kilo-code.md
+33
View File
@@ -0,0 +1,33 @@
# Kilo Code
Kilo Code allows you to configure providers via the UI and can be set up to use AI Gateway.
## Configuration
<div class="tabs">
### OpenAI Compatible
1. Open Kilo Code in VS Code.
1. Go to **Settings**.
1. **Provider**: Select **OpenAI**.
1. **Base URL**: Enter `https://coder.example.com/api/v2/aibridge/openai/v1`.
1. **API Key**: Enter your **[Coder Session Token](../../../admin/users/sessions-tokens.md#generate-a-long-lived-api-token-on-behalf-of-yourself)**.
1. **Model ID**: Enter the model you wish to use (e.g., `gpt-5.2-codex`).
![Kilo Code OpenAI Settings](../../../images/aibridge/clients/kilo-code-openai.png)
### Anthropic
1. Open Kilo Code in VS Code.
1. Go to **Settings**.
1. **Provider**: Select **Anthropic**.
1. **Base URL**: Enter `https://coder.example.com/api/v2/aibridge/anthropic`.
1. **API Key**: Enter your **Coder Session Token**.
1. **Model ID**: Select your desired Claude model.
![Kilo Code Anthropic Settings](../../../images/aibridge/clients/kilo-code-anthropic.png)
</div>
**References:** [Kilo Code Configuration](https://kilocode.ai/docs/ai-providers/openai-compatible)
docs/ai-coder/ai-gateway/clients/mux.md
+96
View File
@@ -0,0 +1,96 @@
# Mux
Mux makes it easy to run parallel coding agents, each with its own isolated workspace, from your browser or desktop; it is open source and provider-agnostic.
Mux can be configured to route OpenAI- and Anthropic-compatible traffic through AI Gateway by setting a custom provider base URL and using a Coder-issued token for authentication.
## Prerequisites
- AI Gateway is enabled on your Coder deployment.
- A **[Coder session token](../../../admin/users/sessions-tokens.md#generate-a-long-lived-api-token-on-behalf-of-yourself)** or long-lived API key.
## Configuration
<div class="tabs">
### OpenAI
1. Open Mux settings (`Cmd+,` / `Ctrl+,`).
2. Go to **Providers** → **OpenAI**.
3. Set **API Key** to your Coder session token.
4. Set **Base URL** to `https://coder.example.com/api/v2/aibridge/openai/v1`.
### Anthropic
1. Open Mux settings (`Cmd+,` / `Ctrl+,`).
2. Go to **Providers** → **Anthropic**.
3. Set **API Key** to your Coder session token.
4. Set **Base URL** to `https://coder.example.com/api/v2/aibridge/anthropic`.
</div>
_Replace `coder.example.com` with your Coder deployment URL._
## Environment variables
Mux reads provider configuration from its settings UI and also from environment variables.
Environment variables are useful in CI or when running Mux inside a Coder workspace.
> [!NOTE]
> Mux treats environment variables as a fallback when a provider is not configured in settings.
> If you have already configured a provider in the UI, clear it (or update it) for env vars to take effect.
```sh
# OpenAI-compatible traffic (GPT, Codex, etc.)
export OPENAI_API_KEY="<your-coder-session-token>"
export OPENAI_BASE_URL="https://coder.example.com/api/v2/aibridge/openai/v1"
# Anthropic-compatible traffic (Claude, etc.)
export ANTHROPIC_API_KEY="<your-coder-session-token>"
export ANTHROPIC_BASE_URL="https://coder.example.com/api/v2/aibridge/anthropic"
```
## Running Mux in a Coder workspace
If you want to run Mux inside a Coder workspace (for example, as a Coder app), you can install it with the [Mux module](https://registry.coder.com/modules/coder/mux) and pre-configure AI Gateway via environment variables on the agent:
```tf
data "coder_workspace" "me" {}
data "coder_workspace_owner" "me" {}
resource "coder_agent" "main" {
# ... other agent configuration
env = {
OPENAI_API_KEY = data.coder_workspace_owner.me.session_token
OPENAI_BASE_URL = "${data.coder_workspace.me.access_url}/api/v2/aibridge/openai/v1"
ANTHROPIC_API_KEY = data.coder_workspace_owner.me.session_token
ANTHROPIC_BASE_URL = "${data.coder_workspace.me.access_url}/api/v2/aibridge/anthropic"
}
}
module "mux" {
source = "registry.coder.com/coder/mux/coder"
version = "~> 1.0" # See the module page for the latest version.
agent_id = coder_agent.main.id
}
```
## Advanced: providers.jsonc
If you prefer a file-based config, edit `~/.mux/providers.jsonc`:
```jsonc
{
"openai": {
"apiKey": "<your-coder-session-token>",
"baseUrl": "https://coder.example.com/api/v2/aibridge/openai/v1"
},
"anthropic": {
"apiKey": "<your-coder-session-token>",
"baseUrl": "https://coder.example.com/api/v2/aibridge/anthropic"
}
}
```
**References:** [Mux provider environment variables](https://mux.coder.com/config/providers#environment-variables)
docs/ai-coder/ai-gateway/clients/opencode.md
+44
View File
@@ -0,0 +1,44 @@
# OpenCode
OpenCode supports both OpenAI and Anthropic models and can be configured to use AI Gateway by setting custom base URLs for each provider.
## Configuration
You can configure OpenCode to connect to AI Gateway by setting the following configuration options in your OpenCode configuration file (e.g., `~/.config/opencode/opencode.json`):
```json
{
"$schema": "https://opencode.ai/config.json",
"provider": {
"anthropic": {
"options": {
"baseURL": "https://coder.example.com/api/v2/aibridge/anthropic/v1"
}
},
"openai": {
"options": {
"baseURL": "https://coder.example.com/api/v2/aibridge/openai/v1"
}
}
}
}
```
## Authentication
To authenticate with AI Gateway, get your **[Coder session token](../../../admin/users/sessions-tokens.md#generate-a-long-lived-api-token-on-behalf-of-yourself)** and replace `<your-coder-session-token>` in `~/.local/share/opencode/auth.json`
```json
{
"anthropic": {
"type": "api",
"key": "<your-coder-session-token>"
},
"openai": {
"type": "api",
"key": "<your-coder-session-token>"
}
}
```
**References:** [OpenCode Documentation](https://opencode.ai/docs/providers/#config)
docs/ai-coder/ai-gateway/clients/roo-code.md
+39
View File
@@ -0,0 +1,39 @@
# Roo Code
Roo Code allows you to configure providers via the UI and can be set up to use AI Gateway.
## Configuration
Roo Code allows you to configure providers via the UI.
<div class="tabs">
### OpenAI Compatible
1. Open Roo Code in VS Code.
1. Go to **Settings**.
1. **Provider**: Select **OpenAI**.
1. **Base URL**: Enter `https://coder.example.com/api/v2/aibridge/openai/v1`.
1. **API Key**: Enter your **[Coder Session Token](../../../admin/users/sessions-tokens.md#generate-a-long-lived-api-token-on-behalf-of-yourself)**.
1. **Model ID**: Enter the model you wish to use (e.g., `gpt-5.2-codex`).
![Roo Code OpenAI Settings](../../../images/aibridge/clients/roo-code-openai.png)
### Anthropic
1. Open Roo Code in VS Code.
1. Go to **Settings**.
1. **Provider**: Select **Anthropic**.
1. **Base URL**: Enter `https://coder.example.com/api/v2/aibridge/anthropic`.
1. **API Key**: Enter your **Coder Session Token**.
1. **Model ID**: Select your desired Claude model.
![Roo Code Anthropic Settings](../../../images/aibridge/clients/roo-code-anthropic.png)
</div>
### Notes
* If you encounter issues with the **OpenAI** provider type, use **OpenAI Compatible** to ensure correct endpoint routing.
* Ensure your Coder deployment URL is reachable from your VS Code environment.
**References:** [Roo Code Configuration Profiles](https://docs.roocode.com/features/api-configuration-profiles#creating-and-managing-profiles)
docs/ai-coder/ai-gateway/clients/vscode.md
+50
View File
@@ -0,0 +1,50 @@
# VS Code
VS Code's native chat can be configured to use AI Gateway with the GitHub Copilot Chat extension's custom language model support.
## Configuration
> [!IMPORTANT]
> You need the **Pre-release** version of the [GitHub Copilot Chat extension](https://marketplace.visualstudio.com/items?itemName=GitHub.copilot-chat) and [VS Code Insiders](https://code.visualstudio.com/insiders/).
1. Open command palette (`Ctrl+Shift+P` or `Cmd+Shift+P` on Mac) and search for _Chat: Open Language Models (JSON)_.
1. Paste the following JSON configuration, replacing `<your-coder-session-token>` with your **[Coder Session Token](../../../admin/users/sessions-tokens.md#generate-a-long-lived-api-token-on-behalf-of-yourself)**:
```json
[
{
"name": "Coder",
"vendor": "customoai",
"apiKey": "your-coder-session-token>",
"models": [
{
"name": "GPT 5.2",
"url": "https://coder.example.com/api/v2/aibridge/openai/v1/chat/completions",
"toolCalling": true,
"vision": true,
"thinking": true,
"maxInputTokens": 272000,
"maxOutputTokens": 128000,
"id": "gpt-5.2"
},
{
"name": "GPT 5.2 Codex",
"url": "https://coder.example.com/api/v2/aibridge/openai/v1/responses",
"toolCalling": true,
"vision": true,
"thinking": true,
"maxInputTokens": 272000,
"maxOutputTokens": 128000,
"id": "gpt-5.2-codex"
}
]
}
]
```
_Replace `coder.example.com` with your Coder deployment URL._
> [!NOTE]
> The setting names may change as the feature moves from pre-release to stable. Refer to the official documentation for the latest setting keys.
**References:** [GitHub Copilot - Bring your own language model](https://code.visualstudio.com/docs/copilot/customization/language-models#_add-an-openaicompatible-model)
docs/ai-coder/ai-gateway/clients/zed.md
+63
View File
@@ -0,0 +1,63 @@
# Zed
Zed IDE supports AI Gateway via its `language_models` configuration in `settings.json`.
## Configuration
To configure Zed to use AI Gateway, you need to edit your `settings.json` file. You can access this by pressing `Cmd/Ctrl + ,` or opening the command palette and searching for "Open Settings".
You can configure both Anthropic and OpenAI providers to point to AI Gateway.
```json
{
"language_models": {
"anthropic": {
"api_url": "https://coder.example.com/api/v2/aibridge/anthropic",
},
"openai": {
"api_url": "https://coder.example.com/api/v2/aibridge/openai/v1",
},
},
// optional settings to set favorite models for the AI
"agent": {
"favorite_models": [
{
"provider": "anthropic",
"model": "claude-sonnet-4-5-thinking-latest"
},
{
"provider": "openai",
"model": "gpt-5.2-codex"
}
],
},
}
```
*Replace `coder.example.com` with your Coder deployment URL.*
> [!NOTE]
> These settings and environment variables need to be configured from client side. Zed currently does not support reading these settings from remote configuration. See this [feature request](https://github.com/zed-industries/zed/discussions/47058) for more details.
## Authentication
Zed requires an API key for these providers. For AI Gateway, this key is your **[Coder Session Token](../../../admin/users/sessions-tokens.md#generate-a-long-lived-api-token-on-behalf-of-yourself)**.
You can set this in two ways:
<div class="tabs">
### Zed UI
1. Open the **Assistant Panel** (right sidebar).
1. Click **Configuration** or the settings icon.
1. Select your provider ("Anthropic" or "OpenAI").
1. Paste your **[Coder Session Token](../../../admin/users/sessions-tokens.md#generate-a-long-lived-api-token-on-behalf-of-yourself)** for the API Key.
### Environment Variables
1. Set `ANTHROPIC_API_KEY` and `OPENAI_API_KEY` to your **[Coder Session Token](../../../admin/users/sessions-tokens.md#generate-a-long-lived-api-token-on-behalf-of-yourself)** in the environment where you launch Zed.
</div>
**References:** [Configuring Zed - Language Models](https://zed.dev/docs/reference/all-settings#language-models)
docs/ai-coder/ai-gateway/index.md
+47
View File
@@ -0,0 +1,47 @@
# AI Gateway
![AI bridge diagram](../../images/aibridge/aibridge_diagram.png)
AI Gateway is a smart gateway for AI. It acts as an intermediary between your users' coding agents / IDEs
and providers like OpenAI and Anthropic. By intercepting all the AI traffic between these clients and
the upstream APIs, AI Gateway can record user prompts, token usage, and tool invocations.
AI Gateway solves 3 key problems:
1. **Centralized authn/z management**: no more issuing & managing API tokens for OpenAI/Anthropic usage.
Users use their Coder session or API tokens to authenticate with `coderd` (Coder control plane), and
`coderd` securely communicates with the upstream APIs on their behalf.
1. **Auditing and attribution**: all interactions with AI services, whether autonomous or human-initiated,
will be audited and attributed back to a user.
1. **Centralized MCP administration**: define a set of approved MCP servers and tools which your users may
use.
> [!NOTE]
> AI Gateway was previously known as "AI Bridge". Some configuration
> options, environment variables, and API paths still use the old name
> and will be updated in a future release.
## When to use AI Gateway
As LLM adoption grows, administrators need centralized auditing, monitoring, and token management. AI Gateway enables organizations to manage AI tooling access for thousands of engineers from a single control plane.
If you are an administrator or devops leader looking to:
- Measure AI tooling adoption across teams or projects
- Establish an audit trail of prompts, issues, and tools invoked
- Manage token spend in a central dashboard
- Investigate opportunities for AI automation
- Uncover high-leverage use cases last
AI Gateway is best suited for organizations facing these centralized management and observability challenges.
## Next steps
- [Set up AI Gateway](./setup.md) on your Coder deployment
- [Configure AI clients](./clients/index.md) to use AI Gateway
- [Configure MCP servers](./mcp.md) for tool access
- [Audit AI sessions](./audit.md)
- [Monitor usage and metrics](./monitoring.md) and [configure data retention](./setup.md#data-retention)
- [Reference documentation](./reference.md)
<children></children>
docs/ai-coder/ai-gateway/mcp.md
+69
View File
@@ -0,0 +1,69 @@
# MCP
> [!WARNING]
> Injected MCP in AI Gateway is deprecated and will be removed in a future release.
[Model Context Protocol (MCP)](https://modelcontextprotocol.io/docs/getting-started/intro) is a mechanism for connecting AI applications to external systems.
AI Gateway can connect to MCP servers and inject tools automatically, enabling you to centrally manage the list of tools you wish to grant your users.
> [!NOTE]
> Only MCP servers which support OAuth2 Authorization are supported currently.
>
> [_Streamable HTTP_](https://modelcontextprotocol.io/specification/2025-06-18/basic/transports#streamable-http) is the only supported transport currently. In future releases we will support the (now deprecated) [_Server-Sent Events_](https://modelcontextprotocol.io/specification/2025-06-18/basic/transports#backwards-compatibility) transport.
AI Gateway makes use of [External Auth](../../admin/external-auth/index.md) applications, as they define OAuth2 connections to upstream services. If your External Auth application hosts a remote MCP server, you can configure AI Gateway to connect to it, retrieve its tools and inject them into requests automatically - all while using each individual user's access token.
For example, GitHub has a [remote MCP server](https://github.com/github/github-mcp-server?tab=readme-ov-file#remote-github-mcp-server) and we can use it as follows.
```bash
CODER_EXTERNAL_AUTH_0_TYPE=github
CODER_EXTERNAL_AUTH_0_CLIENT_ID=...
CODER_EXTERNAL_AUTH_0_CLIENT_SECRET=...
# Tell AI Gateway where it can find this service's remote MCP server.
CODER_EXTERNAL_AUTH_0_MCP_URL=https://api.githubcopilot.com/mcp/
```
See the diagram in [Implementation Details](./reference.md#implementation-details) for more information.
You can also control which tools are injected by using an allow and/or a deny regular expression on the tool names:
```env
CODER_EXTERNAL_AUTH_0_MCP_TOOL_ALLOW_REGEX=(.+_gist.*)
CODER_EXTERNAL_AUTH_0_MCP_TOOL_DENY_REGEX=(create_gist)
```
In the above example, all tools containing `_gist` in their name will be allowed, but `create_gist` is denied.
The logic works as follows:
- If neither the allow/deny patterns are defined, all tools will be injected.
- The deny pattern takes precedence.
- If only a deny pattern is defined, all tools are injected except those explicitly denied.
In the above example, if you prompted your AI model with "list your available github tools by name", it would reply something like:
> Certainly! Here are the GitHub-related tools that I have available:
>
> ```text
> 1. bmcp_github_update_gist
> 2. bmcp_github_list_gists
> ```
AI Gateway marks automatically injected tools with a prefix `bmcp_` ("bridged MCP"). It also namespaces all tool names by the ID of their associated External Auth application (in this case `github`).
## Tool Injection
If a model decides to invoke a tool and it has a `bmcp_` suffix and AI Gateway has a connection with the related MCP server, it will invoke the tool. The tool result will be passed back to the upstream AI provider, and this will loop until the model has all of its required data. These inner loops are not relayed back to the client; all it sees is the result of this loop. See [Implementation Details](./reference.md#implementation-details).
In contrast, tools which are defined by the client (i.e. the [`Bash` tool](https://docs.claude.com/en/docs/claude-code/settings#tools-available-to-claude) defined by _Claude Code_) cannot be invoked by AI Gateway, and the tool call from the model will be relayed to the client, after which it will invoke the tool.
If you have [Coder MCP Server](../mcp-server.md) enabled, as well as have `CODER_AIBRIDGE_INJECT_CODER_MCP_TOOLS=true` set, Coder's MCP tools will be injected into intercepted requests.
### Troubleshooting
- **Too many tools**: should you receive an error like `Invalid 'tools': array too long. Expected an array with maximum length 128, but got an array with length 132 instead`, you can reduce the number by filtering out tools using the allow/deny patterns documented in the [MCP](#mcp) section.
- **Coder MCP tools not being injected**: in order for Coder MCP tools to be injected, the internal MCP server needs to be active. Follow the instructions in the [MCP Server](../mcp-server.md) page to enable it and ensure `CODER_AIBRIDGE_INJECT_CODER_MCP_TOOLS` is set to `true`.
- **External Auth tools not being injected**: this is generally due to the requesting user not being authenticated against the [External Auth](../../admin/external-auth/index.md) app; when this is the case, no attempt is made to connect to the MCP server.
docs/ai-coder/ai-gateway/monitoring.md
+152
View File
@@ -0,0 +1,152 @@
# Monitoring
AI Gateway records the last `user` prompt, token usage, model reasoning, and every tool invocation for each intercepted request. Each capture is tied to a single "interception" that maps back to the authenticated Coder identity, making it easy to attribute spend and behaviour.
![User Prompt logging](../../images/aibridge/grafana_user_prompts_logging.png)
![User Leaderboard](../../images/aibridge/grafana_user_leaderboard.png)
We provide an example Grafana dashboard that you can import as a starting point for your metrics. See [the Grafana dashboard README](https://github.com/coder/coder/blob/main/examples/monitoring/dashboards/grafana/aibridge/README.md).
These logs and metrics can be used to determine usage patterns, track costs, and evaluate tooling adoption.
## Structured Logging
AI Bridge can emit structured logs for every interception event to your
existing log pipeline. This is useful for exporting data to external SIEM or
observability platforms. See [Structured Logging](./setup.md#structured-logging)
in the setup guide for configuration and a full list of record types.
## Exporting Data
AI Gateway interception data can be exported for external analysis, compliance reporting, or integration with log aggregation systems.
### REST API
You can retrieve AI Gateway sessions via the Coder API, with filtering and pagination support.
```sh
curl -X GET "https://coder.example.com/api/v2/aibridge/sessions" \
-H "Coder-Session-Token: $CODER_SESSION_TOKEN"
```
Available query filters:
- `client` - Filter by client name.
<details>
<summary>Possible <code>client</code> values</summary>
> [!NOTE]
> Client classification is done on best effort basis using the `User-Agent` header;
not all clients send these headers in an easily-identifiable manner.
- `Claude Code`
- `Codex`
- `Zed`
- `GitHub Copilot (VS Code)`
- `GitHub Copilot (CLI)`
- `Kilo Code`
- `Mux`
- `Roo Code`
- `Cursor`
- `Unknown`
</details><br>
- `initiator` - Filter by user ID or username
- `provider` - Filter by AI provider (e.g., `openai`, `anthropic`)
- `model` - Filter by model name
- `started_after` - Filter interceptions after a timestamp
- `started_before` - Filter interceptions before a timestamp
See the [API documentation](../../reference/api/aibridge.md) for full details.
### CLI
Export interceptions as JSON using the CLI:
```sh
coder aibridge interceptions list --initiator me --limit 1000
```
You can filter by time range, provider, model, and user:
```sh
coder aibridge interceptions list \
--started-after "2025-01-01T00:00:00Z" \
--started-before "2025-02-01T00:00:00Z" \
--provider anthropic
```
See `coder aibridge interceptions list --help` for all options.
## Data Retention
AI Gateway data is retained for **60 days by default**. Configure the retention
period to balance storage costs with your organization's compliance and analysis
needs.
For configuration options and details, see [Data Retention](./setup.md#data-retention)
in the AI Gateway setup guide.
## Tracing
AI Gateway supports tracing via [OpenTelemetry](https://opentelemetry.io/),
providing visibility into request processing, upstream API calls, and MCP server
interactions.
### Enabling Tracing
AI Gateway tracing is enabled when tracing is enabled for the Coder server.
To enable tracing set `CODER_TRACE_ENABLE` environment variable or
[--trace](https://coder.com/docs/reference/cli/server#--trace) CLI flag:
```sh
export CODER_TRACE_ENABLE=true
```
```sh
coder server --trace
```
### What is Traced
AI Gateway creates spans for the following operations:
| Span Name | Description |
|---------------------------------------------|------------------------------------------------------|
| `CachedBridgePool.Acquire` | Acquiring a request bridge instance from the pool |
| `Intercept` | Top-level span for processing an intercepted request |
| `Intercept.CreateInterceptor` | Creating the request interceptor |
| `Intercept.ProcessRequest` | Processing the request through the bridge |
| `Intercept.ProcessRequest.Upstream` | Forwarding the request to the upstream AI provider |
| `Intercept.ProcessRequest.ToolCall` | Executing a tool call requested by the AI model |
| `Intercept.RecordInterception` | Recording creating interception record |
| `Intercept.RecordPromptUsage` | Recording prompt/message data |
| `Intercept.RecordTokenUsage` | Recording token consumption |
| `Intercept.RecordToolUsage` | Recording tool/function calls |
| `Intercept.RecordInterceptionEnded` | Recording the interception as completed |
| `ServerProxyManager.Init` | Initializing MCP server proxy connections |
| `StreamableHTTPServerProxy.Init` | Setting up HTTP-based MCP server proxies |
| `StreamableHTTPServerProxy.Init.fetchTools` | Fetching available tools from MCP servers |
Example trace of an interception using Jaeger backend:
![Trace of interception](../../images/aibridge/jaeger_interception_trace.png)
### Capturing Logs in Traces
> [!NOTE]
> Enabling log capture may generate a large volume of trace events.
To include log messages as trace events, enable trace log capture
by setting `CODER_TRACE_LOGS` environment variable or using
[--trace-logs](https://coder.com/docs/reference/cli/server#--trace-logs) flag:
```sh
export CODER_TRACE_ENABLE=true
export CODER_TRACE_LOGS=true
```
```sh
coder server --trace --trace-logs
```
docs/ai-coder/ai-gateway/reference.md
+41
View File
@@ -0,0 +1,41 @@
# Reference
## Implementation Details
`coderd` runs an in-memory instance of `aibridged`, whose logic is mostly contained in https://github.com/coder/aibridge. In future releases we will support running external instances for higher throughput and complete memory isolation from `coderd`.
![AI Gateway implementation details](../../images/aibridge/aibridge-implementation-details.png)
## Supported APIs
API support is broken down into two categories:
- **Intercepted**: requests are intercepted, audited, and augmented - full AI Gateway functionality
- **Passthrough**: requests are proxied directly to the upstream, no auditing or augmentation takes place
Where relevant, both streaming and non-streaming requests are supported.
### OpenAI
#### Intercepted
- [`/v1/chat/completions`](https://platform.openai.com/docs/api-reference/chat/create)
- [`/v1/responses`](https://platform.openai.com/docs/api-reference/responses/create)
#### Passthrough
- [`/v1/models(/*)`](https://platform.openai.com/docs/api-reference/models/list)
### Anthropic
#### Intercepted
- [`/v1/messages`](https://docs.claude.com/en/api/messages)
#### Passthrough
- [`/v1/models(/*)`](https://docs.claude.com/en/api/models-list)
## Troubleshooting
To report a bug, file a feature request, or view a list of known issues, please visit our [GitHub repository for AI Gateway](https://github.com/coder/aibridge). If you encounter issues with AI Gateway, please reach out to us via [Discord](https://discord.gg/coder).
docs/ai-coder/ai-gateway/setup.md
+163
View File
@@ -0,0 +1,163 @@
# Setup
AI Gateway runs inside the Coder control plane (`coderd`), requiring no separate compute to deploy or scale. Once enabled, `coderd` runs the `aibridged` in-memory and brokers traffic to your configured AI providers on behalf of authenticated users.
**Required**:
1. A **Premium** license with the [AI Governance Add-On](../ai-governance.md).
1. Feature must be [enabled](#activation) using the server flag
1. One or more [providers](#configure-providers) API key(s) must be configured
## Activation
You will need to enable AI Gateway explicitly:
```sh
export CODER_AIBRIDGE_ENABLED=true
coder server
# or
coder server --aibridge-enabled=true
```
## Configure Providers
AI Gateway proxies requests to upstream LLM APIs. Configure at least one provider before exposing AI Gateway to end users.
<div class="tabs">
### OpenAI
Set the following when routing [OpenAI-compatible](https://coder.com/docs/reference/cli/server#--aibridge-openai-key) traffic through AI Gateway:
- `CODER_AIBRIDGE_OPENAI_KEY` or `--aibridge-openai-key`
- `CODER_AIBRIDGE_OPENAI_BASE_URL` or `--aibridge-openai-base-url`
The default base URL (`https://api.openai.com/v1/`) works for the native OpenAI service. Point the base URL at your preferred OpenAI-compatible endpoint (for example, a hosted proxy or LiteLLM deployment) when needed.
If you'd like to create an [OpenAI key](https://platform.openai.com/api-keys) with minimal privileges, this is the minimum required set:
![List Models scope should be set to "Read", Model Capabilities set to "Request"](../../images/aibridge/openai_key_scope.png)
### Anthropic
Set the following when routing [Anthropic-compatible](https://coder.com/docs/reference/cli/server#--aibridge-anthropic-key) traffic through AI Gateway:
- `CODER_AIBRIDGE_ANTHROPIC_KEY` or `--aibridge-anthropic-key`
- `CODER_AIBRIDGE_ANTHROPIC_BASE_URL` or `--aibridge-anthropic-base-url`
The default base URL (`https://api.anthropic.com/`) targets Anthropic's public API. Override it for Anthropic-compatible brokers.
Anthropic does not allow [API keys](https://console.anthropic.com/settings/keys) to have restricted permissions at the time of writing (Nov 2025).
### Amazon Bedrock
Set the following when routing [Amazon Bedrock](https://coder.com/docs/reference/cli/server#--aibridge-bedrock-region) traffic through AI Gateway:
- `CODER_AIBRIDGE_BEDROCK_REGION` or `--aibridge-bedrock-region`
- `CODER_AIBRIDGE_BEDROCK_ACCESS_KEY` or `--aibridge-bedrock-access-key`
- `CODER_AIBRIDGE_BEDROCK_ACCESS_KEY_SECRET` or `--aibridge-bedrock-access-key-secret`
- `CODER_AIBRIDGE_BEDROCK_MODEL` or `--aibridge-bedrock-model`
- `CODER_AIBRIDGE_BEDROCK_SMALL_FAST_MODEL` or `--aibridge-bedrock-small-fast-model`
> [!NOTE]
> `CODER_AIBRIDGE_BEDROCK_BASE_URL` or `--aibridge-bedrock-base-url` may be used instead of `CODER_AIBRIDGE_BEDROCK_REGION`/`--aibridge-bedrock-region`
if you would like to specify a URL which does not follow the form of `https://bedrock-runtime.<region>.amazonaws.com` - for example if using a
proxy between AI Gateway and AWS Bedrock.
#### Obtaining Bedrock credentials
1. **Choose a region** where you want to use Bedrock.
2. **Generate API keys** in the [AWS Bedrock console](https://us-east-1.console.aws.amazon.com/bedrock/home?region=us-east-1#/api-keys/long-term/create) (replace `us-east-1` in the URL with your chosen region):
- Choose an expiry period for the key.
- Click **Generate**.
- This creates an IAM user with strictly-scoped permissions for Bedrock access.
3. **Create an access key** for the IAM user:
- After generating the API key, click **"You can directly modify permissions for the IAM user associated"**.
- In the IAM user page, navigate to the **Security credentials** tab.
- Under **Access keys**, click **Create access key**.
- Select **"Application running outside AWS"** as the use case.
- Click **Next**.
- Add a description like "Coder AI Gateway token".
- Click **Create access key**.
- Save both the access key ID and secret access key securely.
4. **Configure your Coder deployment** with the credentials:
```sh
export CODER_AIBRIDGE_BEDROCK_REGION=us-east-1
export CODER_AIBRIDGE_BEDROCK_ACCESS_KEY=<your-access-key-id>
export CODER_AIBRIDGE_BEDROCK_ACCESS_KEY_SECRET=<your-secret-access-key>
coder server
```
### Additional providers and Model Proxies
AI Gateway can relay traffic to other OpenAI- or Anthropic-compatible services or model proxies like LiteLLM by pointing the base URL variables above at the provider you operate. Share feedback or follow along in the [`aibridge`](https://github.com/coder/aibridge) issue tracker as we expand support for additional providers.
</div>
> [!NOTE]
> See the [Supported APIs](./reference.md#supported-apis) section below for precise endpoint coverage and interception behavior.
## Data Retention
AI Gateway records prompts, token usage, tool invocations, and model reasoning for auditing and
monitoring purposes. By default, this data is retained for **60 days**.
Configure retention using `--aibridge-retention` or `CODER_AIBRIDGE_RETENTION`:
```sh
coder server --aibridge-retention=90d
```
Or in YAML:
```yaml
aibridge:
retention: 90d
```
Set to `0` to retain data indefinitely.
For duration formats, how retention works, and best practices, see the
[Data Retention](../../admin/setup/data-retention.md) documentation.
## Structured Logging
AI Gateway can emit structured logs for every interception record, making it
straightforward to export data to external SIEM or observability platforms.
Enable with `--aibridge-structured-logging` or `CODER_AIBRIDGE_STRUCTURED_LOGGING`:
```sh
coder server --aibridge-structured-logging=true
```
Or in YAML:
```yaml
aibridge:
structured_logging: true
```
These logs are written to the same output stream as all other `coderd` logs,
using the format configured by
[`--log-human`](../../reference/cli/server.md#--log-human) (default, writes to
stderr) or [`--log-json`](../../reference/cli/server.md#--log-json). For machine
ingestion, set `--log-json` to a file path or `/dev/stderr` so that records are
emitted as JSON.
Filter for AI Gateway records in your logging pipeline by matching on the
`"interception log"` message. Each log line includes a `record_type` field that
indicates the kind of event captured:
| `record_type` | Description | Key fields |
|----------------------|-----------------------------------------|--------------------------------------------------------------------------------|
| `interception_start` | A new intercepted request begins. | `interception_id`, `initiator_id`, `provider`, `model`, `client`, `started_at` |
| `interception_end` | An intercepted request completes. | `interception_id`, `ended_at` |
| `token_usage` | Token consumption for a response. | `interception_id`, `input_tokens`, `output_tokens`, `created_at` |
| `prompt_usage` | The last user prompt in a request. | `interception_id`, `prompt`, `created_at` |
| `tool_usage` | A tool/function call made by the model. | `interception_id`, `tool`, `input`, `server_url`, `injected`, `created_at` |
| `model_thought` | Model reasoning or thinking content. | `interception_id`, `content`, `created_at` |