Nick Vigilante
d40e24cf7d
## Summary Nests the CLI reference by subcommand so the Command Line sidebar collapses from 156 flat entries to 49 top-level verbs. Each page's URL follows the invocation, its page title is the full command as inline code, and its sidebar label is only its own verb as inline code. | | Before | After | | --- | --- | --- | | File | `docs/reference/cli/oauth2-provider_dcr_disable.md` | `docs/reference/cli/oauth2-provider/dcr/disable.md` | | URL | `/reference/cli/oauth2-provider_dcr_disable` | `/reference/cli/oauth2-provider/dcr/disable` | | Page title (front matter) | `oauth2-provider dcr disable` | `` `coder oauth2-provider dcr disable` `` | | Sidebar label (manifest) | `oauth2-provider dcr disable` | `` `disable` `` | Resolves DOCS-140 (coder/coder half). Also fixes the Makefile part of DOCS-1073 / DOCS-912, which this change depends on. ## Changes - `scripts/clidocgen`: pages nest by command path. A command with visible subcommands becomes its directory's `index.md`, so `oauth2-provider/index.md` and `oauth2-provider/dcr/index.md` sit above `oauth2-provider/dcr/disable.md`. Subcommand table links are relative to the parent page. - `scripts/clidocgen`: the manifest "Command Line" section is built from the command tree (nested, sorted by name, hidden commands skipped) instead of a flat list. The redundant `coder` child that pointed back at `reference/cli/index.md` is gone. - `Makefile`: the CLI rule now replaces `docs/reference/cli/` wholesale, so stale pages get removed. The manifest rule now runs `clidocgen -manifest-only` on top of the API staging manifest, so the CLI nav is generated instead of edited by hand. As a result, the nine pages that had no nav entry (`ai-gateway*`, `secret enable/disable`, `server fix-oidc-links`) now have one. - `scripts/configdocgen` and 48 prose pages: links point at the new paths (for example `reference/cli/server.md` is now `reference/cli/server/index.md`). ## How to review/QA this PR - `docs/reference/cli/**/*` (approximately 190 pages): Each of these pages are generated automatically. Some pages appear deleted, and some pages appear new. Every deleted page corresponds to a new page somewhere else, so no information is being deleted. The docs in this section should adhere to the following, so a test in the build preview for each of these items should be satisfactory: - [ ] Each subcommand of a command gets nested as a child instead of being displayed as a flat list. - [ ] Each parent command is stored in an `index.md` file. - [ ] The title of each page inside the page should now feature the full `coder XYZ` command. - [ ] The title of each page inside the page should now be formatted as inline code. - [ ] The title of each page in the sidebar should be the most granular subcommand of the whole command string (if the page's title is `coder ai-gateway keys create`, the sidebar label is `create`). - [ ] The title of each page in the sidebar should now be formatted as inline code. - [ ] The page slug for each page is now the most granular subcommand. - [ ] All links now point to the correct page. - `docs/manifest.json`: The changes here reflect the updated structure of the `docs/reference/cli` section. - `docs/**/*.md` outside of `docs/reference/cli/**/*` (approximately 50 pages): The extent of the changes in these pages are just updating the paths to point to the new page locations. - [ ] The links on the page all point to the correct new path in `docs/reference/cli`. - `scripts/clidocgen` (4 files): This is the Go code responsible for generating everything inside of `docs/reference/cli/**/*`. A typical code review here should be satisfactory. - `scripts/configdocgen` (2 files): these two files just have updates to the raw `.md` path used by `docs/admin/setup/configuration-reference.md` and updates the tests to reflect that. - `Makefile`: This updates the `make` targets for generating the CLI docs to fit the new structure. ## Merge ordering Without the renderer PRs deployed, both docs sites show literal backticks in titles and sidebar labels. Without the redirects, the old flat URLs return 404. 1. Deploy before merging this PR: coder/docs-engine#176 (beta-docs renders these titles as inline code) and coder/coder.com#1111 (classic docs renders them as inline code). Both are safe to ship first, because titles without backticks render as before. 2. Merge this PR. 3. Merge coder/coder.com#1112 (redirects from the old flat URLs) only once this content reaches the mainline release branch that serves unversioned `/docs`. ## Testing - `go test ./scripts/clidocgen/ ./scripts/configdocgen/ ./scripts/docshtmlcheck/`: pass. New tests cover page paths, subcommand links, the nested manifest, and front matter quoting. - `make docs/reference/cli/index.md docs/manifest.json docs/admin/setup/configuration-reference.md`, then a forced rerun: the second run produced no diff. - Checked the manifest: all 164 CLI entries resolve to a file, no page is missing a nav entry, and everything outside "Command Line" is byte-identical. - Relative `.md` links across `docs/`: 0 broken, same as `main`. - `make pre-commit` and `make pre-push`: pass. On earlier pre-push attempts, different unrelated vitest files timed out under host load. Those files pass when run alone, and the final push passed with `PARALLEL_JOBS=1`. <details> <summary>Implementation plan and decision log</summary> **Goal (from DOCS-140 and the requester):** URLs follow the command hierarchy. Pages are titled with the full command, and sidebar entries with the last verb only. Both are formatted as inline code. **Contract with the renderers:** a title wrapped entirely in one pair of backticks (``^`[^`]+`$``) renders as inline code. Plain-text consumers (`<title>`, meta, search, llms corpus) strip the backticks. Page title comes from front matter `title`, and the sidebar label from the manifest node `title`. **Decisions** - A parent command is `dir/index.md` rather than a sibling `dir.md`. The root CLI page is already `index.md`, and both docs-engine and the legacy coder.com renderer map `x/index.md` to `/x`. - The manifest is built from the command tree, not from the written files, so ordering is deterministic and the nesting matches the pages exactly. - The manifest discard bug (DOCS-1073 item 1) is fixed here, because without that fix the generated nested manifest would never reach the tree. DOCS-1073 items 2 through 4 (a prose fix and a hidden-command policy) are out of scope. - `-manifest-only` keeps the existing parallel `make gen` layout: the API rule still owns `_gen/manifest-staging.json`, and CLI regeneration doesn't retrigger swag. - The renderer and redirect work lives in the companion PRs in coder/docs-engine and coder/coder.com. It is linked above and not duplicated here. </details> --- > [!NOTE] > Generated by Coder Agents on behalf of @nickvigilante.
9.8 KiB
title
| title |
|---|
| Scale Testing |
Scaling Coder involves planning and testing to ensure it can handle more load without compromising service. This process encompasses infrastructure setup, traffic projections, and aggressive testing to identify and mitigate potential bottlenecks.
A dedicated Kubernetes cluster for Coder is recommended to configure, host, and manage Coder workloads. Kubernetes provides container orchestration capabilities, allowing Coder to efficiently deploy, scale, and manage workspaces across a distributed infrastructure. This ensures high availability, fault tolerance, and scalability for Coder deployments. Coder is deployed on this cluster using the Helm chart.
For more information about scaling, see our Coder scaling best practices.
Methodology
Our scale tests include the following stages:
-
Prepare environment: create expected users and provision workspaces.
-
SSH connections: establish user connections with agents, verifying their ability to echo back received content.
-
Web Terminal: verify the PTY connection used for communication with Web Terminal.
-
Workspace application traffic: assess the handling of user connections with specific workspace apps, confirming their capability to echo back received content effectively.
-
Dashboard evaluation: verify the responsiveness and stability of Coder dashboards under varying load conditions. This is achieved by simulating user interactions using instances of headless Chromium browsers.
-
Cleanup: delete workspaces and users created in step 1.
Infrastructure and setup requirements
The scale tests runner can distribute the workload to overlap single scenarios based on the workflow configuration:
| T0 | T1 | T2 | T3 | T4 | T5 | T6 | |
|---|---|---|---|---|---|---|---|
| SSH connections | X | X | X | X | |||
| Web Terminal (PTY) | X | X | X | X | |||
| Workspace apps | X | X | X | X | |||
| Dashboard (headless) | X | X | X | X |
This pattern closely reflects how our customers naturally use the system. SSH connections are heavily utilized because they're the primary communication channel for IDEs with VS Code and JetBrains plugins.
The basic setup of scale tests environment involves:
- Scale tests runner (32 vCPU, 128 GB RAM)
- Coder: 2 replicas (4 vCPU, 16 GB RAM)
- Database: 1 instance (2 vCPU, 32 GB RAM)
- Provisioner: 50 instances (0.5 vCPU, 512 MB RAM)
The test is deemed successful if:
- Users did not experience interruptions in their workflows,
coderddid not crash or require restarts, and- No other internal errors were observed.
Traffic Projections
In our scale tests, we simulate activity from 2000 users, 2000 workspaces, and 2000 agents, with two items of workspace agent metadata being sent every 10 seconds. Here are the resulting metrics:
Coder:
- Median CPU usage for
coderd: 3 vCPU, peaking at 3.7 vCPU while all tests are running concurrently. - Median API request rate: 350 RPS during dashboard tests, 250 RPS during Web Terminal and workspace apps tests.
- 2000 agent API connections with latency: p90 at 60 ms, p95 at 220 ms.
- on average 2400 Web Socket connections during dashboard tests.
Provisionerd:
- Median CPU usage is 0.35 vCPU during workspace provisioning.
Database:
- Median CPU utilization is 80%, with a significant portion dedicated to writing workspace agent metadata.
- Memory utilization averages at 40%.
write_ops_countbetween 6.7 and 8.4 operations per second.
Available reference architectures
Hardware recommendation
Control plane: coderd
To ensure stability and reliability of the Coder control plane, it's essential to focus on node sizing, resource limits, and the number of replicas. We recommend referencing public cloud providers such as AWS, GCP, and Azure for guidance on optimal configurations. A reasonable approach involves using scaling formulas based on factors like CPU, memory, and the number of users.
While the minimum requirements specify 1 CPU core and 2 GB of memory per coderd replica, we recommend that you allocate additional resources depending on the workload size to ensure deployment stability.
CPU and memory usage
Enabling agent stats collection (optional) may increase memory consumption.
Enabling direct connections between users and workspace agents (apps or SSH traffic) can help prevent an increase in CPU usage. It is recommended to keep this option enabled unless there are compelling reasons to disable it.
Inactive users do not consume Coder resources.
Scaling formula
When determining scaling requirements, consider the following factors:
1 vCPU x 2 GB memoryfor every 250 users: A reasonable formula to determine resource allocation based on the number of users and their expected usage patterns.- API latency/response time: Monitor API latency and response times to ensure optimal performance under varying loads.
- Average number of HTTP requests: Track the average number of HTTP requests to gauge system usage and identify potential bottlenecks. The number of proxied connections: For a very high number of proxied connections, more memory is required.
HTTP API latency
For a reliable Coder deployment dealing with medium to high loads, it's important that API calls for workspace/template queries and workspace build operations respond within 300 ms. However, API template insights calls, which involve browsing workspace agent stats and user activity data, may require more time. Moreover, Coder API exposes WebSocket long-lived connections for Web Terminal (bidirectional), and Workspace events/logs (unidirectional).
If the Coder deployment expects traffic from developers spread across the globe, be aware that customer-facing latency might be higher because of the distance between users and the load balancer. Fortunately, the latency can be improved with a deployment of Coder workspace proxies.
Node Autoscaling
We recommend disabling the autoscaling for coderd nodes. Autoscaling can cause
interruptions for user connections, see
Autoscaling for more details.
Control plane: Workspace Proxies
When scaling workspace proxies, follow the
same guidelines as for coderd above:
1 vCPU x 2 GB memoryfor every 250 users.- Disable autoscaling.
Control plane: provisionerd
Each external provisioner can run a single concurrent workspace build. For example, running 10 provisioner containers will allow 10 users to start workspaces at the same time.
By default, the control plane runs 3 built-in provisioner daemons, but the
Premium Coder release allows for running external provisioners to separate the
load caused by workspace provisioning on the coderd nodes.
Scaling formula
When determining scaling requirements, consider the following factors:
1 vCPU x 1 GB memory x 2 concurrent workspace build: A formula to determine resource allocation based on the number of concurrent workspace builds, and standard complexity of a Terraform template. Rule of thumb: the more provisioners are free/available, the more concurrent workspace builds can be performed.
Node Autoscaling
Autoscaling provisioners is not an easy problem to solve unless it can be predicted when a number of concurrent workspace builds increases.
We recommend disabling autoscaling and adjusting the number of provisioners to developer needs based on the workspace build queuing time.
Data plane: Workspaces
To determine workspace resource limits and keep the best developer experience for workspace users, administrators must be aware of a few assumptions.
- Workspace pods run on the same Kubernetes cluster, but possibly in a different namespace or on a separate set of nodes.
- Workspace limits (per workspace user):
- Evaluate the workspace utilization pattern. For instance, web application development does not require high CPU capacity at all times, but will spike during builds or testing.
- Evaluate minimal limits for single workspace. Include in the calculation requirements for Coder agent running in an idle workspace - 0.1 vCPU and 256 MB. For instance, developers can choose between 0.5-8 vCPUs, and 1-16 GB memory.
Scaling formula
When determining scaling requirements, consider the following factors:
1 vCPU x 2 GB memory x 1 workspace: A formula to determine resource allocation based on the minimal requirements for an idle workspace with a running Coder agent and occasional CPU and memory bursts for building projects.
Node Autoscaling
Workspace nodes can be set to operate in autoscaling mode to mitigate the risk of prolonged high resource utilization.
One approach is to scale up workspace nodes when total CPU usage or memory consumption reaches 80%. Another option is to scale based on metrics such as the number of workspaces or active users. It's important to note that as new users onboard, the autoscaling configuration should account for ongoing workspaces.
Scaling down workspace nodes to zero is not recommended, as it will result in longer wait times for workspace provisioning by users. However, this may be necessary for workspaces with special resource requirements (e.g. GPUs) that incur significant cost overheads.