Nick Vigilante
d40e24cf7d
## Summary Nests the CLI reference by subcommand so the Command Line sidebar collapses from 156 flat entries to 49 top-level verbs. Each page's URL follows the invocation, its page title is the full command as inline code, and its sidebar label is only its own verb as inline code. | | Before | After | | --- | --- | --- | | File | `docs/reference/cli/oauth2-provider_dcr_disable.md` | `docs/reference/cli/oauth2-provider/dcr/disable.md` | | URL | `/reference/cli/oauth2-provider_dcr_disable` | `/reference/cli/oauth2-provider/dcr/disable` | | Page title (front matter) | `oauth2-provider dcr disable` | `` `coder oauth2-provider dcr disable` `` | | Sidebar label (manifest) | `oauth2-provider dcr disable` | `` `disable` `` | Resolves DOCS-140 (coder/coder half). Also fixes the Makefile part of DOCS-1073 / DOCS-912, which this change depends on. ## Changes - `scripts/clidocgen`: pages nest by command path. A command with visible subcommands becomes its directory's `index.md`, so `oauth2-provider/index.md` and `oauth2-provider/dcr/index.md` sit above `oauth2-provider/dcr/disable.md`. Subcommand table links are relative to the parent page. - `scripts/clidocgen`: the manifest "Command Line" section is built from the command tree (nested, sorted by name, hidden commands skipped) instead of a flat list. The redundant `coder` child that pointed back at `reference/cli/index.md` is gone. - `Makefile`: the CLI rule now replaces `docs/reference/cli/` wholesale, so stale pages get removed. The manifest rule now runs `clidocgen -manifest-only` on top of the API staging manifest, so the CLI nav is generated instead of edited by hand. As a result, the nine pages that had no nav entry (`ai-gateway*`, `secret enable/disable`, `server fix-oidc-links`) now have one. - `scripts/configdocgen` and 48 prose pages: links point at the new paths (for example `reference/cli/server.md` is now `reference/cli/server/index.md`). ## How to review/QA this PR - `docs/reference/cli/**/*` (approximately 190 pages): Each of these pages are generated automatically. Some pages appear deleted, and some pages appear new. Every deleted page corresponds to a new page somewhere else, so no information is being deleted. The docs in this section should adhere to the following, so a test in the build preview for each of these items should be satisfactory: - [ ] Each subcommand of a command gets nested as a child instead of being displayed as a flat list. - [ ] Each parent command is stored in an `index.md` file. - [ ] The title of each page inside the page should now feature the full `coder XYZ` command. - [ ] The title of each page inside the page should now be formatted as inline code. - [ ] The title of each page in the sidebar should be the most granular subcommand of the whole command string (if the page's title is `coder ai-gateway keys create`, the sidebar label is `create`). - [ ] The title of each page in the sidebar should now be formatted as inline code. - [ ] The page slug for each page is now the most granular subcommand. - [ ] All links now point to the correct page. - `docs/manifest.json`: The changes here reflect the updated structure of the `docs/reference/cli` section. - `docs/**/*.md` outside of `docs/reference/cli/**/*` (approximately 50 pages): The extent of the changes in these pages are just updating the paths to point to the new page locations. - [ ] The links on the page all point to the correct new path in `docs/reference/cli`. - `scripts/clidocgen` (4 files): This is the Go code responsible for generating everything inside of `docs/reference/cli/**/*`. A typical code review here should be satisfactory. - `scripts/configdocgen` (2 files): these two files just have updates to the raw `.md` path used by `docs/admin/setup/configuration-reference.md` and updates the tests to reflect that. - `Makefile`: This updates the `make` targets for generating the CLI docs to fit the new structure. ## Merge ordering Without the renderer PRs deployed, both docs sites show literal backticks in titles and sidebar labels. Without the redirects, the old flat URLs return 404. 1. Deploy before merging this PR: coder/docs-engine#176 (beta-docs renders these titles as inline code) and coder/coder.com#1111 (classic docs renders them as inline code). Both are safe to ship first, because titles without backticks render as before. 2. Merge this PR. 3. Merge coder/coder.com#1112 (redirects from the old flat URLs) only once this content reaches the mainline release branch that serves unversioned `/docs`. ## Testing - `go test ./scripts/clidocgen/ ./scripts/configdocgen/ ./scripts/docshtmlcheck/`: pass. New tests cover page paths, subcommand links, the nested manifest, and front matter quoting. - `make docs/reference/cli/index.md docs/manifest.json docs/admin/setup/configuration-reference.md`, then a forced rerun: the second run produced no diff. - Checked the manifest: all 164 CLI entries resolve to a file, no page is missing a nav entry, and everything outside "Command Line" is byte-identical. - Relative `.md` links across `docs/`: 0 broken, same as `main`. - `make pre-commit` and `make pre-push`: pass. On earlier pre-push attempts, different unrelated vitest files timed out under host load. Those files pass when run alone, and the final push passed with `PARALLEL_JOBS=1`. <details> <summary>Implementation plan and decision log</summary> **Goal (from DOCS-140 and the requester):** URLs follow the command hierarchy. Pages are titled with the full command, and sidebar entries with the last verb only. Both are formatted as inline code. **Contract with the renderers:** a title wrapped entirely in one pair of backticks (``^`[^`]+`$``) renders as inline code. Plain-text consumers (`<title>`, meta, search, llms corpus) strip the backticks. Page title comes from front matter `title`, and the sidebar label from the manifest node `title`. **Decisions** - A parent command is `dir/index.md` rather than a sibling `dir.md`. The root CLI page is already `index.md`, and both docs-engine and the legacy coder.com renderer map `x/index.md` to `/x`. - The manifest is built from the command tree, not from the written files, so ordering is deterministic and the nesting matches the pages exactly. - The manifest discard bug (DOCS-1073 item 1) is fixed here, because without that fix the generated nested manifest would never reach the tree. DOCS-1073 items 2 through 4 (a prose fix and a hidden-command policy) are out of scope. - `-manifest-only` keeps the existing parallel `make gen` layout: the API rule still owns `_gen/manifest-staging.json`, and CLI regeneration doesn't retrigger swag. - The renderer and redirect work lives in the companion PRs in coder/docs-engine and coder/coder.com. It is linked above and not duplicated here. </details> --- > [!NOTE] > Generated by Coder Agents on behalf of @nickvigilante.
6.4 KiB
title
| title |
|---|
| Setup |
By default, AI Gateway runs inside the Coder control plane (coderd) and requires no separate compute.
In embedded mode, coderd runs the gateway in memory and brokers traffic to your configured AI providers on behalf of authenticated users.
If AI traffic needs dedicated compute, independent scaling, or a separate network endpoint, you can deploy AI Gateway as a standalone service.
Provider records are managed through the dashboard or API and stored in the database. Provider environment variables, flags, and YAML options no longer seed the database. Refer to Database management of providers for details.
Activation
The AI Gateway feature must be enabled in the Coder deployment configuration before embedded or standalone gateway instances can serve authenticated traffic.
AI Gateway is enabled by default as of v2.34.
export CODER_AI_GATEWAY_ENABLED=true
coder server
# or
coder server --ai-gateway-enabled=true
A standalone process does not read CODER_AI_GATEWAY_ENABLED from its own environment.
However, this setting must remain enabled on coderd.
It is required for gateway key management endpoints to work and for standalone replicas to connect to the control plane.
Configure Providers
Configure at least one provider before exposing AI Gateway to end users.
Providers are deployment-scoped. Add them from the dashboard or the
AI Providers API. Changes take effect
without restarting coderd.
Dashboard
- Navigate to Admin settings > AI
- Select Providers
- Select Add provider
- Select the provider type
- Enter a unique lowercase name, the upstream endpoint, and the credentials
- Save the provider
Each provider gets its own AI Gateway route at
/api/v2/ai-gateway/<provider-name>/.
Note
Provider names must be unique and use lowercase, hyphen-separated identifiers such as
anthropic-corporazure-openai. Once deleted, another provider may reuse the name.
Open an existing provider to rotate credentials, update its endpoint, or
disable it without restarting coderd.
API Dumps
AI Gateway can dump provider request and response pairs to disk for debugging.
Configure the dump directory with --ai-gateway-dump-dir or
CODER_AI_GATEWAY_DUMP_DIR:
coder server --ai-gateway-dump-dir=/var/lib/coder/ai-gateway-dumps
Or in YAML:
ai_gateway:
api_dump_dir: /var/lib/coder/ai-gateway-dumps
This top-level setting replaces the previous per-provider DUMP_DIR field.
For each provider, AI Gateway writes dumps under <base>/<provider_name>, where
<base> is the configured dump directory and <provider_name> is the provider
instance name used in the route. For example, a provider named anthropic-corp
with /var/lib/coder/ai-gateway-dumps configured writes to
/var/lib/coder/ai-gateway-dumps/anthropic-corp.
Sensitive headers are redacted before dumps are written. Leave the value empty to disable dumping.
Each standalone gateway replica accepts the same API dump settings and writes optional dumps to its own local disk.
Warning
API dumps are intended for short diagnostic sessions only. Dump files contain raw request and response data, which may include proprietary or sensitive information such as prompts, completions, and tool inputs. Protect the target directory and disable dumping when diagnostics are complete.
Data Retention
AI Gateway records prompts, token usage, tool invocations, and model reasoning for auditing and monitoring purposes. By default, this data is retained for 60 days.
Configure retention using --ai-gateway-retention or CODER_AI_GATEWAY_RETENTION:
coder server --ai-gateway-retention=90d
Or in YAML:
ai_gateway:
retention: 90d
Set to 0 to retain data indefinitely.
For duration formats, how retention works, and best practices, see the Data Retention documentation.
Structured Logging
AI Gateway can emit structured logs for every interception record, making it straightforward to export data to external SIEM or observability platforms.
Enable with --ai-gateway-structured-logging or CODER_AI_GATEWAY_STRUCTURED_LOGGING:
coder server --ai-gateway-structured-logging=true
Or in YAML:
ai_gateway:
structured_logging: true
These logs are written to the same output stream as all other coderd logs,
using the format configured by
--log-human (default, writes to
stderr) or --log-json. For machine
ingestion, set --log-json to a file path or /dev/stderr so that records are
emitted as JSON.
This setting belongs to coderd.
A standalone gateway does not consume it.
Filter for AI Gateway records in your logging pipeline by matching on the
"interception log" message. Each log line includes a record_type field that
indicates the kind of event captured:
record_type |
Description | Key fields |
|---|---|---|
interception_start |
A new intercepted request begins. | interception_id, initiator_id, provider, model, client, started_at |
interception_end |
An intercepted request completes. | interception_id, ended_at |
token_usage |
Token consumption for a response. | interception_id, input_tokens, output_tokens, created_at |
prompt_usage |
The last user prompt in a request. | interception_id, prompt, created_at |
tool_usage |
A tool/function call made by the model. | interception_id, tool, input, server_url, injected, created_at |
model_thought |
Model reasoning or thinking content. | interception_id, content, created_at |