opensource/coder
3
0
Fork 0
mirror of https://github.com/coder/coder.git synced 2026-09-24 15:04:27 +08:00
Code Issues Packages Projects Releases Wiki Activity

docs: add standalone AI Gateway docs (#27592)

Browse Source 
Documents standalone AI Gateway deployment, Gateway key authentication,
monitoring, and the updated embedded vs standalone topology in the AI
Gateway docs.

---------

Co-authored-by: Cian Johnston <cian@coder.com>
This commit is contained in:
Paweł Banaszewski
2026-07-29 19:38:22 +02:00
committed by GitHub
co-authored by Cian Johnston
parent 6c42309ccb
commit 18128b7b52
18 changed files with 773 additions and 136 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/admin/infrastructure/architecture.md
+9 -1
View File
@@ -138,7 +138,15 @@ as OpenAI and Anthropic. Users authenticate through Coder instead of managing se
provider API keys. All prompts, token usage, and tool invocations are recorded
for compliance and cost tracking.
Learn more: [AI Gateway](../../ai-coder/ai-gateway/index.md)
AI Gateway supports 2 deployment topologies:
- **Embedded:** `coderd` runs the AI Gateway data plane in the same process.
- **Standalone:** AI Gateway runs outside `coderd`, as replicas that serve AI traffic and send requests directly to upstream providers.
Standalone replicas hold no durable state. `coderd` is the source of truth and the only component that writes AI Gateway state to the database.
Each replica maintains a control connection to `coderd` for Coder API key validation, provider configuration, and AI session recording, and becomes unready when that connection is unavailable.
Refer to [AI Gateway](../../ai-coder/ai-gateway/index.md) and [standalone deployment](../../ai-coder/ai-gateway/standalone.md) for configuration and operational guidance.
### Agent Firewall
docs/admin/infrastructure/validated-architectures/index.md
+10
View File
@@ -123,6 +123,16 @@ or helper scripts.
Please note that the Registry is a hosted service and isn't available for
offline use.
### AI Gateway
[AI Gateway](../../../ai-coder/ai-gateway/index.md) proxies AI provider traffic
and records each AI session. It runs inside `coderd` by default, and can also
run as a [standalone deployment](../../../ai-coder/ai-gateway/standalone.md)
that scales independently of the control plane. Size replicas from your own AI
request volume and `CODER_AI_GATEWAY_MAX_CONCURRENCY`. For the chart's resource
requests and autoscaling defaults, refer to the
[AI Gateway Helm chart README](https://github.com/coder/coder/blob/main/helm/ai-gateway/README.md).
## Kubernetes Infrastructure
Kubernetes is the recommended, and supported platform for deploying Coder in the