opensource/coder
3
0
Fork 0
mirror of https://github.com/coder/coder.git synced 2026-10-31 11:02:50 +08:00
Code Issues Packages Projects Releases Wiki Activity

docs: address AI Gateway documentation review feedback (#28309)

Browse Source 
Applies the review feedback from #27794. Prose only.

---------

Co-authored-by: Nick Vigilante <nickvigilante@users.noreply.github.com>
This commit is contained in:
Paweł Banaszewski
2026-08-23 19:42:48 +00:00
committed by GitHub
co-authored by Nick Vigilante
parent c9a1e2bcee
commit 4ce8e8ed56
17 changed files with 178 additions and 193 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/admin/infrastructure/architecture.md
+7 -6
View File
@@ -133,17 +133,18 @@ AI-powered development within Coder workspaces.
### AI Gateway
AI Gateway is a centralized gateway that sits between coding agents and LLM providers such
as OpenAI and Anthropic. Users authenticate through Coder instead of managing separate
provider API keys. All prompts, token usage, and tool invocations are recorded
for compliance and cost tracking.
AI Gateway is a centralized feature that sits between coding agents and LLM providers such as OpenAI and Anthropic.
Users authenticate through Coder instead of managing separate provider API keys.
All prompts, token usage, and tool invocations are recorded for compliance and cost tracking.
AI Gateway supports 2 deployment topologies:
- **Embedded:** `coderd` runs the AI Gateway data plane in the same process.
- **Standalone:** AI Gateway runs outside `coderd`, as replicas that serve AI traffic and send requests directly to upstream providers.
- **Standalone:** gateway replicas run outside `coderd`, serve AI traffic, and send requests directly to upstream providers.
Standalone replicas hold no durable state. `coderd` is the source of truth and the only component that writes AI Gateway state to the database.
Standalone replicas have no authoritative database state.
They keep ephemeral provider snapshots, caches, and metrics in memory, and can optionally write API dumps to local disk.
`coderd` is the source of truth and the only component that writes AI Gateway state to the database.
Each replica maintains a control connection to `coderd` for Coder API key validation, provider configuration, and AI session recording, and becomes unready when that connection is unavailable.
Refer to [AI Gateway](../../ai-coder/ai-gateway/index.md) and [standalone deployment](../../ai-coder/ai-gateway/standalone.md) for configuration and operational guidance.
docs/admin/infrastructure/validated-architectures/index.md
+4 -7
View File
@@ -125,13 +125,10 @@ offline use.
### AI Gateway
[AI Gateway](../../../ai-coder/ai-gateway/index.md) proxies AI provider traffic
and records each AI session. It runs inside `coderd` by default, and can also
run as a [standalone deployment](../../../ai-coder/ai-gateway/standalone.md)
that scales independently of the control plane. Size replicas from your own AI
request volume and `CODER_AI_GATEWAY_MAX_CONCURRENCY`. For the chart's resource
requests and autoscaling defaults, refer to the
[AI Gateway Helm chart README](https://github.com/coder/coder/blob/main/helm/ai-gateway/README.md).
[AI Gateway](../../../ai-coder/ai-gateway/index.md) proxies AI provider traffic and records each AI session.
It runs inside `coderd` by default and can also run as a [standalone deployment](../../../ai-coder/ai-gateway/standalone.md) that scales independently of the control plane.
Size replicas based on your AI request volume and `CODER_AI_GATEWAY_MAX_CONCURRENCY`.
For the chart's resource requests and autoscaling defaults, refer to the [AI Gateway Helm chart README](../../../../helm/ai-gateway/README.md).
## Kubernetes Infrastructure
docs/admin/integrations/multiple-kube-clusters.md
+1 -1
View File
@@ -46,7 +46,7 @@ coder:
readOnly: true
```
[Upgrade Coder](../../install/kubernetes.md#upgrading-coder-via-helm) with these
[Upgrade Coder](../../install/kubernetes.md#upgrade-coder-via-helm) with these
new values.
### VM control plane
docs/admin/integrations/prometheus.md
+3 -6
View File
@@ -72,12 +72,9 @@ scrape_configs:
apps: "coder"
```
If you run a [standalone AI Gateway](../../ai-coder/ai-gateway/standalone.md),
each replica exports its own metrics on its own listener. Its Helm chart uses the
same `0.0.0.0:2112` default as the `coder` chart, but sets up no scrape
discovery. Refer to
[AI Gateway monitoring](../../ai-coder/ai-gateway/monitoring.md#kubernetes-discovery)
for more details.
If you run a [standalone AI gateway](../../ai-coder/ai-gateway/standalone.md), each replica exports its own metrics on its own listener.
Its Helm chart uses the same `0.0.0.0:2112` default as the `coder` chart, but sets up no scrape discovery.
Refer to [AI gateway monitoring](../../ai-coder/ai-gateway/monitoring.md#kubernetes-discovery) for more details.
To use the Kubernetes Prometheus operator to scrape metrics, you will need to
create a `ServiceMonitor` in your Coder deployment namespace. The following is