mirror of
https://github.com/coder/coder.git
synced 2026-09-22 05:05:20 +08:00
a3a0079bd2fcb0f29a0ea8e21ada1bc8599e42b5
11
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Nick Vigilante
|
0a7bb80a1e |
feat: make CLI/API doc generators emit front-matter metadata (Phase 2) (#27246)
## Summary
Phase 2 of the H1 → front-matter migration (`DOCS-483`; parent
`DOCS-477`). Makes the two reference-doc generators emit per-page
metadata as YAML front matter instead of a leading `# H1`, so generated
pages are self-describing and `make gen` stops reverting migrated pages
(Phase 3).
Phase 1 (`DOCS-482`) made the coder.com renderers prefer a front-matter
`title` (manifest fallback).
> [!NOTE]
> Rebased onto `main` and fully regenerated, and updated across two
rounds of Coder Agents Review — see **Review follow-ups** below.
## Changes
- **`scripts/clidocgen/command.tpl` + `gen.go` + `main.go`** — front
matter now carries `title` (from `fullName`) and `description` (from the
command's `Short`), and the leading `# H1` is dropped. The CLI index
page's front matter is taken from the manifest `Command Line` route
(title/description/icon_path).
- **`scripts/apidocgen/postprocess/main.go`** — reads the manifest and,
at write time, injects front matter carrying each section's `title` plus
any curated `description`, `state`, and `icon_path`. The API index
page's front matter is taken from the manifest `REST API` route.
- **`scripts/docgenenv`** (new shared code) — one `YAMLScalar`
front-matter escaper, one `Route`/`Manifest` schema +
`LoadManifest`/`FindRoute`, and one `FrontMatter(Route)` emitter, all
imported by both generators (no duplicated helpers, types, or emitters).
- Regenerated all **166 CLI + 31 API** reference pages.
### Metadata → front matter, and what stays in the manifest
Every *per-page* manifest field is mirrored into the page's front
matter: `title`, `description`, `state`, `icon_path`. The **structural**
fields stay in `manifest.json`:
- `children` — the nav tree (explicitly out of scope).
- `path` — the manifest's pointer to the file; a page carrying its own
path is redundant/error-prone, so it's treated like `children`.
The fields are **duplicated** into front matter and **`manifest.json` is
left unchanged**, so this is a **no-op for rendering today** (coder.com
strips front matter for `llms`, and Algolia + the renderer read only
`title`). Removing the fields from the manifest is the natural
follow-up, gated on the renderer reading them from front matter first.
### Why the API side changes the postprocessor, not the `.dot` templates
The issue text suggested editing
`scripts/apidocgen/markdown-template/*`. I deliberately did **not**,
because the postprocessor derives each page's **filename, section title,
and manifest route** from the leading `# {name}` line
(`extractSectionName`). Emitting front matter from the template would
break that extraction. Instead the widdershins templates still emit `#
{name}`, the postprocessor reads it (and now verifies it), and then
swaps the heading for a front-matter block as each section is written.
## Review follow-ups (Coder Agents Review)
### Round 1 — addressed in `e53d5e03` (all threads resolved)
- **CRF-1 / CRF-4** — de-duplicated the escaper and the
`route`/`manifest` schema + traversal into `scripts/docgenenv` (shared
by both generators).
- **CRF-2** — `YAMLScalar` now quotes YAML-reserved scalars
(`true/false/null/…`, numbers); no current value is affected.
- **CRF-3** — added unit tests: a `YAMLScalar` round-trip, `FindRoute`,
and `prependFrontMatter`.
- **CRF-5** — the CLI and API **index** pages now mirror their manifest
route's title/description/icon_path instead of a hardcoded
`coder`/`API`, fixing a rendered-heading regression (`REST API`/`Command
Line` were being overwritten).
- **CRF-6** — dropped the dead `#login` anchor in
`docs/support/support-bundle.md` (the migrated `login.md` no longer
mints that heading anchor).
- **CRF-7 / CRF-8 / CRF-11** — renamed to `prependFrontMatter`, switched
to `bytes.Cut`, and it now strips the first line only when it is the `#
{name}` heading (`extractSectionName` errors otherwise).
- **CRF-9** — removed the orphan `docs/reference/api/chat.md` (not in
the manifest, not linked; the real page is `chats.md`).
- **CRF-10** — the metadata read and the manifest rewrite now share one
`FindRoute` traversal.
- **CRF-13** — moot under squash-merge; this branch is a single
scopeless commit.
- **CRF-15** — the pre-existing `sort.Slice`/`slices.IsSorted`
comparator is left as-is per the review (out of scope; safe today
because section names are unique).
### Round 2 — addressed in `ee796e7107` (all threads resolved)
- **CRF-16** (P1) — removed three em-dashes from new doc comments (the
only `make lint` failure on the prior head); the emdash gate is green.
- **CRF-17 / CRF-18** — unified front-matter emission into one shared
`docgenenv.FrontMatter(Route)`, used by the API postprocessor directly
and by `command.tpl` via a `frontMatter` template func. This retires the
hand-written template YAML and the
`indexTitle`/`indexDescription`/`indexIconPath` closures, so a new
front-matter field is wired in one place, and it gives the CLI index the
`state` arm it previously lacked. Verified byte-identical: a full CLI +
API regen produces zero page changes.
- **CRF-19** — CLI child sort switched to `slices.SortFunc` +
`cmp.Compare` (typed comparator).
- **CRF-20** — reworded the `prependFrontMatter` comment:
`extractSectionName`'s fail-fast is the load-bearing guard; the prefix
check is a defensive backstop.
- **CRF-21** — added `icon_path`/`state` coverage in `docgenenv`'s
`TestFrontMatter/AllFields` (the branch the index page relies on,
previously at 0%).
- **CRF-22** — `YAMLScalar` no longer emits a trailing-space value as a
bare scalar (YAML strips it on read, so it would not round-trip); added
test coverage.
- **CRF-24** — the shared emitter removed the duplicated `cliIndexRoute`
doc comment; the rationale now lives in one place.
- **CRF-23** (Phase 3, out of scope here) — noted: the API generator
wipes and regenerates `reference/api/` from the manifest, so removing
curated metadata from the manifest in Phase 3 needs another source first
(a generator that preserves existing front matter, or metadata carried
alongside the swagger annotations).
- **Process (Mafu-san)** — the verification set below now leads with
`make lint`, the mandatory CI gate that the earlier list omitted.
## Cross-repo dependency
**Resolved — this PR no longer has a hard merge-ordering gate** (CRF-14
was right; the earlier "must merge after #968" note was stale).
The coder.com surfaces that would otherwise leak raw front matter from
`coder/coder` `main` are already front-matter-aware on merged PRs:
- **coder.com#964** (`DOCS-554`, llms-full.txt corpus + Algolia) —
**merged**.
- **coder.com#974** (`DOCS-574`, the `.md` proxy twin + `llms.txt` index
titles) — **merged**.
coder.com#968 (`DOCS-577`) was re-scoped to only the renderer
route-metadata generalization; it's a no-op on today's corpus and its
own description confirms the "deploy before the generators" constraint
no longer applies (that was driven by the llms corpus, now in #964).
Worth a final confirmation that #964/#974 are **deployed** before merge,
but there's no branch/PR ordering blocker left.
## Verification & evidence
AI was the primary author of this PR (see disclosure below); per the [AI
Contribution
Guidelines](https://coder.com/docs/about/contributing/AI_CONTRIBUTING)
here is manual verification.
- `make lint` (golangci-lint + the emdash gate) passes; `go build` / `go
vet` / `go test` are clean for the generators + `scripts/docgenenv`;
`pnpm check-docs` passes.
- `swagger.json`, `docs.go`, and `manifest.json` are **unchanged** —
metadata is duplicated into front matter; command/section names and
routes did not move.
- The diff is purely additive front matter
(`title`/`description`/`state`/`icon_path`) + the leading H1 removal; no
body reflow. A full CLI + API regen produces **zero** page changes
beyond the two index pages.
<details>
<summary>Terminal evidence</summary>
CLI `description` from the command's `Short` (`YAMLScalar` quotes when
needed, e.g. a `Short` with a colon):
```md
---
title: server
description: Start a Coder server
---
```
API pages inherit curated manifest metadata (only Agents/Chats have any
today):
```md
---
title: Chats
description: "REST endpoints for Coder Agents Chats API (programmatic agent sessions)."
state:
- early access
---
```
Diff scope + "no body changes" proof (uses an explicit `base..HEAD`
range, so it actually tests the claim):
```
$ git diff --shortstat origin/main
210 files changed, 1447 insertions(+), 344 deletions(-)
# = 166 CLI + 31 API reference pages + generators + scripts/docgenenv
# swagger.json / docs.go / manifest.json: NOT modified
# Every removed line under docs/reference is a leading "# H1"; nothing else:
$ git diff origin/main..HEAD -- docs/reference/ | grep '^-' | grep -v '^---' | grep -v '^-# '
(empty)
$ pnpm check-docs
Summary: 0 error(s)
```
</details>
Linear: DOCS-483
> This PR was created with AI assistance (Coder Agents).
|
||
|
Ethan
|
0ac23e3ee1 |
feat: add per-template Coder Agents access control (#27285)
Relates to CODAGT-713 Depends on #27284 This makes the per-template `agents_allowed` field authoritative in the API and chatd. It adds optional create and metadata update fields with the intended default and omission semantics, supports `agents-allowed:` template search, includes the value in telemetry, and makes `list_templates`, `read_template`, and `create_workspace` read the template row directly. Existing-workspace retries remain idempotent, and blocked same-organisation templates return an actionable message. The experimental `/template-allowlist` routes remain temporarily because the shipped AI Settings page still calls them, but they no longer control chatd enforcement. #27514 moves that page to per-template metadata, #27515 removes the legacy storage, routes, SDK types, and utility, #27517 adds the CLI flags, and #27518 updates the platform controls documentation for the per-template model, directly addressing CRF-5 and CRF-6. The stack is intended to merge as a unit. |
||
|
Jeremy RuppelandCoder Agent
|
51ac968d5a |
feat: wire up Template Builder session telemetry endpoint (#27124)
`TemplateBuilderSession` telemetry types and telemetry-server ingestion were added in earlier PRs (#25082, coder/coder-telemetry-server#41), but no code ever produced session events. This adds the missing producer. **Backend**: `POST /api/v2/templatebuilder/sessions` reports wizard entry and compose completion events directly via `api.Telemetry.Report()`, using the same inline pattern as `NetworkEvents` and `UserTailnetConnections`. No database migration or `createSnapshot()` changes needed. RBAC requires `policy.ActionCreate` on `ResourceTemplate.AnyOrganization()`, matching the compose endpoint. **Frontend**: The template builder wizard fires `wizard_entry` on page mount and `compose_completion` on create success or failure. A client-generated session ID (UUID) correlates the two events for the same wizard visit, enabling precise funnel analysis and abandonment detection in BigQuery. Duration is tracked via `Date.now()` in the wizard state. Closes https://linear.app/codercom/issue/DEVEX-599 <details> <summary>Implementation plan</summary> ## Root Cause Analysis The DEVEX-599 ticket diagnosis suggested missing DB tables, queries, and `eg.Go` blocks. That diagnosis assumes the DB-backed periodic snapshot path is required. It is not. Investigation shows two telemetry reporting patterns in the codebase: 1. **DB-backed periodic snapshots** (`createSnapshot()` with `eg.Go` blocks): Used for durable entities like workspaces, templates, users. 2. **Direct inline reporting** (`api.Telemetry.Report(&telemetry.Snapshot{...})`): Used for ephemeral events like `NetworkEvents`, `UserTailnetConnections`, `CLIInvocations`. Template builder sessions are ephemeral events, so the direct inline reporting pattern is the correct fit. ## Backend Changes - `codersdk/templatebuilder.go`: `TemplateBuilderSessionRequest` type with `SessionID`, `EventType` enum, `TemplateBuilderSession()` client method - `coderd/coderd.go`: Route registration in `/templatebuilder` group - `coderd/templatebuilder_handler.go`: Handler with RBAC check, request validation, session ID fallback, and inline telemetry report - `coderd/templatebuilder_handler_test.go`: Tests for wizard entry, compose completion, invalid event type, disabled feature, and member RBAC rejection ## Frontend Changes - `site/src/api/api.ts`: `recordTemplateBuilderSession` API method - `site/src/api/queries/templateBuilder.ts`: React Query mutation - `site/src/pages/TemplateBuilder/wizardState.ts`: `sessionId` and `enteredAt` fields, `createWizardState()` factory for per-mount initialization - `site/src/pages/TemplateBuilder/TemplateBuilderPageView.tsx`: `sessionId` prop, `useReducer` initializer form - `site/src/pages/TemplateBuilder/TemplateBuilderPage.tsx`: Telemetry calls for wizard entry (on mount) and compose completion (on create success/failure) </details> > 🤖 Generated by Coder Agents --------- Co-authored-by: Coder Agent <agent@coder.com> |
||
|
Nick Vigilante
|
c84aa564ba |
docs: normalize code-fence languages for Shiki compatibility (#27161)
Normalizes non-standard code-fence language tags across `docs/**` so a strict highlighter (Shiki, used by Fumadocs) won't fail the build on an unrecognized language, and unifies redundant synonym tags onto one canonical form per language. The current renderer (Speed-Highlight) detects the language from the code content, not the fence label, so this drift wasn't visible until now. ## Changes - `hcl` -> `tf` (199 fences, including indented ones nested in numbered/bulleted lists). Shiki ships `hcl` and `terraform` as two distinct grammars (not aliases); every `hcl`-tagged fence in `docs/**` is actually Terraform resource/data/provider syntax, so the more specific `terraform` grammar is correct for all of them. `tf` is Shiki's own alias for that grammar, and it's also what GitHub's own markdown renderer resolves to the same HCL/Terraform highlighting. - `pwsh`/`powershell` -> `ps1`. Both `ps` and `ps1` are registered PowerShell aliases in Shiki, but on GitHub's renderer only `.ps1` is a registered file extension (`.ps` isn't), so `ps1` renders identically to `powershell` there today while bare `ps` would silently lose highlighting. - `env` -> `dotenv` (a dedicated Shiki grammar for `KEY=VALUE` files) - `text`/`output`/`none`/`url` -> `txt`. Same built-in plain-text fallback either way, just shorter. - `Dockerfile` -> `dockerfile` (lowercase) - `bash`/`shell` -> `sh` (732 fences). Shiki and GitHub both alias all three to a single shell grammar; this was already the style guide's stated preference, just not enforced across the existing corpus until now. - `markdown` -> `md` (4 fences). Alias of the same grammar in both Shiki and GitHub. - `jsonc` -> `json` (1 fence). The block has no comments or trailing commas, so it doesn't need the comments-capable grammar. - `ts` -> `tsx` (2 fences, `docs/about/contributing/frontend.md`). Verified the actual content tokenizes identically under both grammars, and a sibling block in the same file already needs `tsx` for real JSX, so unifying to one tag is safe for this file. Documented a caveat: `tsx` mis-tokenizes the legacy angle-bracket type-assertion syntax (`<Type>value`), which is invalid in real `.tsx` files anyway, so use `value as Type` instead. - `yml` -> `yaml` (1 fence) - Updated `docs/.style/style-guide/formatting.md` to document all canonical tags `promql` (2 fences) and `caddyfile` (2 fences) are left as-is. Shiki doesn't bundle a grammar for either, so they need a custom grammar registration when the site adopts Shiki, rather than degrading to `txt`. Tracked as follow-up work under DOCS-118 and [DOCS-544](https://linear.app/codercom/issue/DOCS-544/vendor-a-local-promql-grammar-for-shiki-syntax-highlighting) (promql). Does not touch `offlinedocs/`. Linear: [DOCS-476](https://linear.app/codercom/issue/DOCS-476/normalize-docs-code-fence-languages-de-risk-shikifumadocs) <details> <summary>How the fence tags were verified</summary> Each tag was tested against a real `shiki@latest` highlighter instance (`codeToHtml`/`codeToTokens`) and cross-checked against GitHub's `@wooorm/starry-night` grammar sources (the renderer that actually displays these `.md` files today, in repo browsing and PR diffs), since that's what determines whether brevity is safe before Shiki adoption: ```text FAIL env -- Language `env` is not included in this bundle. FAIL Dockerfile -- Language `Dockerfile` is not included in this bundle. FAIL promql -- Language `promql` is not included in this bundle. FAIL caddyfile -- Language `caddyfile` is not included in this bundle. FAIL pwsh -- Language `pwsh` is not included in this bundle. FAIL output -- Language `output` is not included in this bundle. ``` `hcl` doesn't error in Shiki, since it's a real grammar, but that's exactly the trap: it was silently rendering every fence with the generic HCL grammar instead of the Terraform-specific one. Every `hcl`-tagged fence in `docs/**` was manually checked against `origin/main` and is genuinely Terraform content. For `ts`/`tsx`, tokenizing the actual doc content confirmed identical output under both grammars; a synthetic test with the legacy angle-bracket cast syntax confirmed `tsx` degrades on that specific construct, which the style guide now calls out. The first normalization pass only matched fence tags at column 0 (`^```tag$`), missing tags indented inside numbered/bulleted lists. A follow-up pass caught the remaining occurrences at any indentation level. </details> --- *This PR description and the underlying changes were prepared with Coder Agents assistance.* |
||
|
Jon Ayers
|
4cfed1b3ed | feat: plumb time_til_autostop_notify template field (#26439) | ||
|
Jeremy Ruppel
|
10717572ac |
feat: show template prerequisites in builder UI (#26523)
Surface base template prerequisites to admins before they create a
template in the Template Builder wizard.
Today, template prerequisites (Docker socket setup, Kubernetes auth, AWS
IAM policies) are only visible in the registry README after import.
Admins hit opaque provisioner errors and have to hunt for docs. This
change extracts the prerequisites from the README and serves them via
the API so the frontend can display them inline.
## How it works
Each base template README uses HTML comment markers (`<!--
prerequisites:start -->` / `<!-- prerequisites:end -->`) to delimit the
prerequisites section. At boot time, the base catalog loader reads the
README, extracts the content between markers via `strings.Index`, and
caches both the full README and the prerequisites string.
The prerequisites are served via a new `prerequisites` field on `GET
/api/v2/templatebuilder/bases`. The full README is included in the
composed template tar bundle and stored as the template version readme.
## Changes
- Add `README.md` with prerequisite markers to
`coderd/templatebuilder/bases/{docker,kubernetes,aws-linux}/`
- New `ExtractPrerequisites()` in `prerequisites.go` using literal
string matching
- `bases.go`: load README at boot, fail loudly if missing, extract
prerequisites
- `compose.go`: include README in `ComposeResult` and tar bundle
- `codersdk`: add `Prerequisites` field to `TemplateBuilderBase`
- Handler: populate prerequisites in bases response, set readme on
template version
<details>
<summary>Implementation notes</summary>
- Prerequisites extraction uses `strings.Index` for exact literal marker
matching; no regex or AST parser needed since we control the markers.
- YAML frontmatter is deliberately retained in the stored README. The
frontend `TemplateDocsPage` already strips it at render time via
`front-matter`.
- The prerequisite markers are HTML comments, invisible in rendered
markdown.
- The `RejectsMissingReadme` test enforces that every base template must
include a README.
- AWS Linux prerequisites span two H2 sections (`## Prerequisites` and
`## Required permissions / policy`), which is why heading-based parsing
was rejected in favor of explicit markers.
*Generated with the assistance of an AI coding agent. Reviewed by
@jeremyruppel.*
</details>
Relates to https://linear.app/codercom/issue/DEVEX-446
|
||
|
Jeremy Ruppel
|
87de6dc23e | feat: add base template variables to API (#26425) | ||
|
Jeremy Ruppel
|
de31c7c18e |
feat: add TemplateBuilderCreateTemplate SDK types and client method (#26360)
Adds `POST /api/v2/templatebuilder/compose/template`, a synchronous endpoint that composes a template from a base and modules, validates it via a provisioner import job, and creates the template in a single request. The handler composes terraform files, bundles them as a tar, inserts the file with hash-based dedup, creates a template version with an import job, waits up to 2 minutes for the job to complete, classifies errors for known failure modes (network-unreachable registry, DNS failures), then creates the template on success. Canceled and failed jobs return appropriate error responses. Also adds `hclwrite.Format` to composed terraform output for canonical HCL formatting. Closes https://linear.app/codercom/issue/DEVEX-279 <details> <summary>Implementation notes</summary> - SDK types and client method in `codersdk/templatebuilder.go` with validation tags matching the standard template creation path (`template_display_name`, `lt=128`) - `ClassifyProvisionerError` in `coderd/templatebuilder/errors.go` detects DNS, connection refused, i/o timeout, and TLS handshake failures and returns actionable messages - `waitForProvisionerJob` polls with a ramp-up interval schedule (100ms, 200ms, 500ms, then 1s steady) and accepts an `onUpdate` callback for future SSE streaming - Audit logging for both template and template version creation - TOCTOU name uniqueness: early check for fast feedback, DB unique constraint catch for the race window (returns 409, not 500) - Swagger annotations for all error responses (400, 404, 409, 504) </details> > 🤖 Generated by Coder Agents |
||
|
Jeremy Ruppel
|
b61b62f4b3 |
feat: add POST /api/v2/templatebuilder/compose endpoint (#26351)
> [!NOTE] > This PR was authored by Coder Agents on behalf of @jeremyruppel. Part 4 of DEVEX-277 (POST /api/v2/templatebuilder/compose). Adds the HTTP handler, route wiring, and integration tests for the compose endpoint. The handler accepts a JSON request with a base template ID and optional modules with variable overrides, renders them via `Compose`/`BundleTar`, and returns the tar archive directly with `Content-Type: application/x-tar`. The registry URL comes from the deployment config (`CODER_TEMPLATE_BUILDER_REGISTRY_URL`). RBAC uses `policy.ActionCreate` on `rbac.ResourceTemplate.AnyOrganization()`. Integration tests cover: base-only compose, base with modules, unknown base/module errors, missing base template ID, and feature-disabled 404. |
||
|
Jeremy Ruppel
|
9a6e348f5d |
feat: add GET /api/v2/templatebuilder/modules endpoint (#26117)
Implement `GET /api/v2/templatebuilder/modules`, which returns the filtered list of modules available for a given base template. Reads from the bundled catalog via `LoadModules()` and applies OS-compatibility filtering based on the `base` query param. Computed variables (e.g. `agent_id`) are excluded from the API response at the `ToSDK()` conversion boundary since they are wired automatically by the builder. The `Computed` field is removed from the SDK type. Adds `CompatibleWithOS()` to `ModuleManifest` for OS filtering. Returns 400 for unknown base IDs and 404 when the template builder is disabled. Depends on #26116 > [!NOTE] > This PR was authored by Coder Agents on behalf of @jeremyruppel. |
||
|
Jeremy Ruppel
|
776fbfa748 |
feat: add GET /api/v2/templatebuilder/bases endpoint (#26116)
Implement `GET /api/v2/templatebuilder/bases`, which returns the list of base templates available in the template builder. Reads from the bundled catalog by cross-referencing `templatebuilder.BaseTemplateIDs()` with `examples.List()`, enriching each entry with the OS from the `exampleID -> OS` map. The endpoint is gated behind the template builder feature flag (returns 404 when disabled) and requires `policy.ActionRead` on `rbac.ResourceTemplate`. Depends on #26115 > [!NOTE] > This PR was authored by Coder Agents on behalf of @jeremyruppel. |