The `coderd_chatd_message_count` histogram's current max bucket of 128
is being hit in production. This increases the exponential bucket count
from 8 to 11, extending coverage from `1..128` to `1..1024`.
Before: `1, 2, 4, 8, 16, 32, 64, 128`
After: `1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024`
Co-authored-by: blink-so[bot] <211532188+blink-so[bot]@users.noreply.github.com>
Context files (AGENTS.md) and skills were only fetched from the
workspace on the first turn or when the agent changed. On subsequent
turns, stale content from persisted messages was used. This meant that
if AGENTS.md or skills were modified on the workspace between turns, the
agent wouldn't see the changes until the user created a new chat.
## Changes
- Extract `fetchWorkspaceContext` from `persistInstructionFiles` to
allow fetching workspace context without persisting
- On subsequent turns, re-fetch fresh context from the workspace instead
of reading stale persisted content; falls back to persisted messages if
the workspace dial fails
- Update `ReloadMessages` callback to re-derive instruction and skills
from reloaded database messages after compaction, instead of using
captured closure variables
- Add `formatSystemInstructionsFromParts` helper to build system
instructions directly from agent parts without requiring separate
OS/directory params
- Add tests for the new helper
<details><summary>Implementation Notes</summary>
### Root cause
In `runChat`, the `else if hasContextFiles` branch (subsequent turns)
called `instructionFromContextFiles(messages)` which read stale content
from persisted DB messages. The `ReloadMessages` callback
(post-compaction) also used captured `instruction`/`skills` closure
variables from the start of the turn, never re-deriving them.
### Approach
1. **Extract `fetchWorkspaceContext`** — Pure refactor of the fetch-only
part of `persistInstructionFiles` (agent connection, context config
retrieval, content sanitization, metadata stamping). Returns parts +
skills without persisting.
2. **Subsequent turns**: Instead of reading from persisted messages,
launch a `g2` goroutine that calls `fetchWorkspaceContext` to get fresh
context from the workspace. Falls back gracefully to persisted messages
if the workspace is unreachable.
3. **ReloadMessages**: Re-derive `instruction` from
`instructionFromContextFiles(reloadedMsgs)` and `skills` from
`skillsFromParts(reloadedMsgs)` using the freshly loaded messages, with
fallback to captured values if the reloaded messages don't contain
context (e.g. compacted away).
</details>
> 🤖 Generated by Coder Agents
## Summary
Adds `--ai-gateway-allow-byok` deployment option to control whether
users can use Bring Your Own Key (BYOK) mode with AI Gateway.
When disabled (`--ai-gateway-allow-byok=false`), BYOK requests are
rejected with a 403 and a message directing the admin to enable the
flag. Centralized key authentication works regardless of this setting.
Defaults to `true` (BYOK allowed).
---------
Co-authored-by: Danny Kopping <danny@coder.com>
Removes the claim that users can fork a chat to explore a different
direction — this is not a supported feature and the reference is
misleading.
---
*PR generated with Coder Agents*
## Summary
Fixes the "See all templates" link overlapping template items in the New
Workspace dropdown.
## Root cause
Two compounding issues:
1. **`OverflowY` className was being overwritten, not merged.** The
component spread `...attrs` (which included the caller's `className`)
onto the div, silently replacing its own base classes
(`overflow-y-auto`, `shrink`, `w-full`). This meant the template list
never scrolled independently.
2. **`PopoverContent` has `overflow-y-auto` in its base styles.** With
the inner `OverflowY` not scrolling, the *entire popover* became the
scroll container. The "See all templates" footer was part of that
scrollable flow and overlapped template rows as the user scrolled.
## Fix
- **`OverflowY`**: Destructure `className` explicitly and merge it with
the base classes using `cn()` so `overflow-y-auto` and `shrink` are
always preserved.
- **`PopoverContent`**: Add `overflow-hidden flex flex-col` to make it a
non-scrolling flex container. Only the `OverflowY` child scrolls.
- **`OverflowY` usage**: Add `min-h-0` so the flex child can shrink
below its content size when the popover's available height is
constrained.
## Screenshot
<img width="1460" height="1072" alt="image"
src="https://github.com/user-attachments/assets/9b519f2d-9806-44ca-a354-12248de36952"
/>
> 🤖 Generated with [Coder Agents](https://coder.com/agents)
## Problem
`resolveDeploymentSystemPrompt` was called inside `InTx` closures in
both `CreateChat` (`coderd/x/chatd/chatd.go`) and
`createChildSubagentChatWithOptions` (`coderd/x/chatd/subagent.go`).
That method uses `p.db` (the root store) internally to call
`GetChatSystemPromptConfig`, which requires a second DB pool checkout
while the transaction already holds one connection.
Under concurrent chat creation load (e.g., the chat scaletest at 4800
chats), this causes pool starvation: every in-flight create holds one
connection and blocks waiting for another, leading to `idle in
transaction` pileups and cascading timeouts across the entire coderd DB
pool — including unrelated background work like prebuild metrics and the
chat acquire loop.
## Fix
Move the `resolveDeploymentSystemPrompt` call before `p.db.InTx(...)` in
both call sites. The system prompt config is a read-only
deployment-level setting that does not need transactional consistency
with the chat insert, so fetching it before the transaction is both safe
and preferable (it also shortens transaction lifetime).
## Backporting
The `CreateChat` instance of this bug is also present on `release/2.32`
(`coderd/x/chatd/chatd.go` line 907). The `subagent.go` instance is not
— the child-subagent-chat creation path with its own `InTx` was added
after the branch cut.
This should be backported, but because this is only in the chat creation
path, and that's not typically hit with a great deal of concurrency in
the real world, I don't think an urgent patch for 2.32 is necessary.
## Lint gap
The existing `InTx` ruleguard rule in `scripts/rules.go` catches direct
outer-store usage (`p.db.GetFoo()`) and passing the outer store as a
function argument inside `InTx` closures, but it explicitly cannot catch
indirect access through receiver methods like
`p.resolveDeploymentSystemPrompt()` — the rule documents this blind spot
at line 273. Catching this class of bug would require interprocedural
analysis (following the callee's body to see if it touches `p.db`),
which is beyond what ruleguard's AST pattern matching can express. We're
considering a lightweight custom `go/analysis` analyzer (similar to
`paralleltestctx`) that does 1-level same-package callee inspection to
detect this pattern. In the meantime, this PR adds guidance to
`AGENTS.md` so AI reviewers can flag the pattern during code review.
This change reuses the authenticated subject's existing organization
membership information during chat creation instead of issuing an
`OrganizationMembers` query.
The current query is still correct, so this is not required for
correctness. However, `workspaceapps` already answers the same question
more cheaply from the request's RBAC subject. This extracts that logic
into `rbac.Subject.HasOrganizationMembership` and reuses it in both
places, removing an extra database lookup from chat creation without
changing the authorization behavior.
I'm currently debugging a Coder agents scaletest regression where a run
on April 2, 2026 with 4800 concurrent chat creations passed, while the
same run on April 15, 2026 does not. We could stagger chat creation to
reduce the burst, but I'd rather understand why this bottleneck appeared
in the first place so we can keep making small hot-path improvements
like this one instead of only smoothing over the symptom.
## Summary
Follows up on https://github.com/coder/coder/pull/24032
Adds a BYOK compatibility table to the AI Gateway client configuration
page, showing which clients support personal API keys and provider
subscriptions through AI Gateway.
We can simplify by merging related columns:
- Personal API Key (OpenAI) and Personal API Key (Anthropic) → Personal
API Key
- ChatGPT Subscription and Claude Subscription → Subscription (Claude
Pro/Max, ChatGPT Plus/Pro)
`NOTE`: This is displayed immediately after the existing Compatibility
table.
<img width="864" height="474" alt="image"
src="https://github.com/user-attachments/assets/644c5a7c-a9fe-454c-9112-3e3db268afc8"
/>
OSV-Scanner currently causes the scheduled security workflow to fail
whenever it reports findings, which also triggers the Slack failure
notification.
Treat exit code 1 as a successful scan with uploaded SARIF results, and
only fail the job when OSV-Scanner itself errors.
_Disclaimer: produced by Claude Opus 4.6_
Adds a `coder_build_info` metric which allows operators to see which
versions of Coder are currently running.
---------
Signed-off-by: Danny Kopping <danny@coder.com>
Restore the container vulnerability scan in the security workflow by
replacing the removed Trivy job with OSV-Scanner.
This keeps the existing image build, SARIF upload, artifact upload, and
Slack failure notification flow, while pinning OSV-Scanner to the latest
release and using the current `--output-file` flag.
Addresses review findings from #23827 that were added post-merge:
- Persisted attachments now store `organizationId`; mismatched orgs
pruned on restore
- Workspace selection reconciliation: stale IDs from previous orgs
dropped via derived `effectiveWorkspaceId`
- Org picker uses `permittedOrganizations()` for RBAC-aware filtering
- Org picker hidden when user belongs to only one org
- Ref-sync `useEffect` replaced with `useEffectEvent`
- `CreateWorkspace()` and `ListTemplates()` take `organizationID` and
`db` as required function parameters instead of optional struct fields —
compiler enforces them, removes scattered nil guards
- Cross-org template check in `CreateWorkspace` is now unconditional
- `ListTemplates` org-scoping filter now has test coverage
- `setupChatInfra` comment fixed; test helpers use params structs
instead of positional UUIDs
- Enterprise test documents that org admin only sees own chats (handler
hardcodes `OwnerID` — future work needs sidebar UI before lifting that
restriction)
> 🤖
_Disclaimer: produced mostly by Claude Opus 4.6 following detailed
planning._
## Summary
- Support multiple instances of the same AI Bridge provider type via
indexed env vars (`CODER_AIBRIDGE_PROVIDER_<N>_<KEY>`), following the
`CODER_EXTERNAL_AUTH_<N>_<KEY>` pattern
- Existing single-provider env vars (`CODER_AIBRIDGE_OPENAI_KEY`, etc.)
continue to work unchanged
- Setting both a legacy env var and an indexed provider with the same
name errors at startup to prevent silent misconfiguration
- Mark legacy provider fields (`OpenAI`, `Anthropic`, `Bedrock`) as
deprecated in `AIBridgeConfig` in favor of `Providers`
## Example
```sh
CODER_AIBRIDGE_PROVIDER_0_TYPE=anthropic
CODER_AIBRIDGE_PROVIDER_0_NAME=anthropic-corp
CODER_AIBRIDGE_PROVIDER_0_KEY=sk-ant-corp-xxx
CODER_AIBRIDGE_PROVIDER_0_BASE_URL=https://llm-proxy.internal.example.com/anthropic
CODER_AIBRIDGE_PROVIDER_1_TYPE=anthropic
CODER_AIBRIDGE_PROVIDER_1_NAME=anthropic-direct
CODER_AIBRIDGE_PROVIDER_1_KEY=sk-ant-direct-yyy
```
Each instance is routed by name:
- /api/v2/aibridge/**anthropic-corp**/v1/messages
- /api/v2/aibridge/**anthropic-direct**/v1/messages
Closes
[AIGOV-157](https://linear.app/codercom/issue/AIGOV-157/spike-to-understand-if-there-is-a-simple-way-to-handle-multi-api-key)
---------
Signed-off-by: Danny Kopping <danny@coder.com>
We noticed during higher active workspace counts that the agent
connection metric, generated via a query to the database, would report a
relatively high amount of agents as disconnected. Somewhere between 5
and 20%. However, other metrics such as # of websocket connections would
suggest that all agent connections are healthy.
Looking at the `Agents` function in prometheus metrics, plus the query
execution time (not accounting for actual database RT time) revealed
that this reporting of agents as disconnected was almost certainly false
positives due to clock drift in the way we're generating the metric
values. At 10k metrics, with a p50 of 2ms and p99 of 5ms, the entire
`agents` function could take upwards of 50s to execute. Because we were
doing a query/database RT to query th apps for each agent individually,
and grabbing a `time.Now` value on each iteration of that loop, it's
likely the portion of agents that were reported as disconnected were
those that had last heartbeat the furthest in the past.
The fix here is to set a consistent `now` before fetching agent data to
avoid clock drift inflating the inactive timeout comparison, and replace
the per-agent app query N+1 with a single batched lookup to prevent loop
execution time from pushing agents over the disconnected threshold.
Signed-off-by: Callum Styan <callumstyan@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
The `_gen/bin/clidocgen` binary only declared `scripts/clidocgen/*.go`
as prerequisites. Since it reflects over the full CLI tree (227
transitive internal packages via `enterprise/cli` → `cli/` → `codersdk/`
→ …), any change to CLI flags, SDK structs, or command definitions could
alter its output — but Make would keep serving the stale binary until it
was manually deleted (or `-B` was passed).
This caused a recurring developer-facing bug: after merging main (or
rebasing onto new CLI/SDK changes), the pre-commit hook would use the
stale binary, commit wrong docs, `make gen` would see no diff (same
stale binary), and CI would fail because it builds fresh.
Add `$(GO_SRC_FILES)` and the embedded `command.tpl` to the prerequisite
list so Make invalidates the binary whenever its inputs change. Move
`FIND_EXCLUSIONS` and `GO_SRC_FILES` above the helper-binary block so
the variable is defined before first use.
## Summary
Follows up on https://github.com/coder/coder/pull/24032
Renames "Coder session token" to "Coder API token" in AI Gateway client
documentation pages.
Also renames the `CODER_SESSION_TOKEN` env var to `CODER_API_TOKEN` in
Codex CLI examples and Copilot proxy configuration.
Note: "Coder session token" is still used in some parts of the
documentation where it make sense.
---------
Co-authored-by: Susana Ferreira <susana@coder.com>
Previously we reserved some env vars that may collide with AI gateway.
These were incomplete and take away flexibility from the user, which
we're prioritizing in the first iteration of the feature.
## Summary
Add `coderd_agents_first_connection_seconds` histogram metric that
records the
duration from workspace agent creation to first connection. This fills
an
observability gap — provisioner job timings and startup script metrics
exist,
but the agent connection phase (which can take several minutes) was not
exposed
to Prometheus.
Closes https://github.com/coder/coder/issues/21282
## Changes
- **`coderd/prometheusmetrics/prometheusmetrics.go`** — Define and
register a
`HistogramVec` in the existing `Agents()` polling loop. Observe
`first_connected_at - created_at` exactly once per agent via a
deduplication
map, pruned each tick to prevent unbounded memory growth.
- **`coderd/prometheusmetrics/prometheusmetrics_test.go`** — Update
`TestAgents`
to set `first_connected_at` on the test agent and assert the histogram
is
collected with correct labels, sample count, and sample sum.
- **`docs/admin/integrations/prometheus.md`**,
**`scripts/metricsdocgen/generated_metrics`** —
Auto-generated documentation updates from `make gen`.
## Metric details
| Property | Value |
|---|---|
| Name | `coderd_agents_first_connection_seconds` |
| Type | histogram |
| Labels | `template_name`, `agent_name`, `username`, `workspace_name` |
| Buckets | 1s, 10s, 30s, 1m, 2m, 5m, 10m, 30m, 1h |
## Example PromQL
```promql
# P95 agent connection time by template
histogram_quantile(0.95,
sum(rate(coderd_agents_first_connection_seconds_bucket[1h])) by (le, template_name)
)
```
<details>
<summary>Implementation notes</summary>
### Design decisions
- **Histogram over gauge**: Enables `histogram_quantile()` for
percentile queries.
- **Observe in `Agents()` polling loop**: All required data is already
fetched by
`GetWorkspaceAgentsForMetrics()` — no new DB queries.
- **Dedup via `map[uuid.UUID]struct{}`**: Prevents re-observing the same
agent
across polling ticks. Pruned each cycle to bound memory.
- **Buckets**: Aligned with
`coderd_provisionerd_workspace_build_timings_seconds`
range (1s–1h).
### Overhead at scale (100k active workspaces)
The deduplication map (`observedFirstConnection`) and per-tick pruning
map
(`currentAgentIDs`) are both `map[[16]byte]struct{}`. At 100k agents:
- **Memory**: ~2.25 MB persistent + ~2.25 MB transient per tick = **~4.5
MB peak**.
- **CPU**: ~25 ms of map operations per tick (one tick per minute) =
**<0.05% of one core**.
Both are negligible relative to the existing cost of the `Agents()` loop
(the DB
query, per-agent `GetWorkspaceAppsByAgentID` calls, and coordinator node
lookups
dominate).
</details>
> 🤖 Generated by Coder Agents
This was originally added because it was present in `env` output in
dogfood, but it's specifically injected in the dogfood template so it
doesn't make sense to deny across the board for a secret environment
variable name.
Three SQL queries (`GetUserGroupSpendLimit`,
`ResolveUserChatSpendLimit`, `GetUserChatSpendInPeriod`) aggregated chat
spend limits and usage globally across all organizations. A restrictive
group limit in org A would bleed into org B.
## Changes
- Add `organization_id` parameter to all three SQL queries in
`coderd/database/queries/chats.sql`
- When nil UUID is passed, queries fall back to global behavior
(backward compat for HTTP dashboard endpoints)
- When real org ID is passed, limits and spend are scoped to that
organization
- Thread `organizationID` through `ResolveUsageLimitStatus` →
`checkUsageLimit` → all chatd call sites
- Update dbauthz wrappers for new param structs
- HTTP endpoints (`chatCostSummary`, `getMyChatUsageLimitStatus`) pass
`uuid.Nil` with TODO for future org-scoped UI
- Add `TestResolveUsageLimitStatus_OrgScoped` with 5 test cases covering
org isolation, nil-UUID fallback, spend scoping, and user override
priority
Closescoder/internal#1466
> 🤖
The vitest process hung after all 2132 story tests passed because
leftover refetchInterval polls kept the Node.js event loop alive.
Components that set per-query refetchInterval override the
QueryClient default, causing HTTP requests through vite's proxy
to localhost:3000 (no backend) that never resolve cleanly.
Three fixes:
- preview.tsx: disable all automatic refetching defaults and cancel
in-flight queries on story unmount via useEffect cleanup
- storybook.tsx: save/restore the original window.WebSocket in the
withWebSocket decorator, clear pending timers in close()
- vite.config.mts: add explicit testTimeout, hookTimeout, bail, and
retry settings to the storybook vitest project
Also fix 5 story files that imported from @testing-library/react
instead of storybook/test.
Bumps the Go toolchain from 1.25.8 to 1.25.9 across `go.mod`, the shared
setup-go action, and the dogfood image.
This keeps local builds, CI, and containerized workflows aligned on the
latest patch release, including the updated Go tarball checksum in the
Dockerfile.
> This PR was authored by Mux on behalf of Mike.
Chats sharing one workspace (e.g. sibling subagents) all wrote to
`/home/coder/PLAN.md`, causing plan file collisions. This change derives
a unique plan path per chat from the workspace home directory and chat
ID.
## Changes
* `write_file`, `edit_files`, and `propose_plan` reject any `plan.md`
variant (case-insensitive) at the workspace home root, with a clear
error pointing to the chat-specific path.
* Root chats receive a `<plan-file-path>` block inlined in the main
system prompt with the concrete path.
* Prompt and tool descriptions no longer hardcode `/home/coder/PLAN.md`.
* Plan path handling is POSIX-only (forward-slash), relying on the
contract that workspace agent paths are normalized before reaching
chatd.
* Updated `ProposePlanTool.stories.tsx` to use per-chat path examples.
* Full test coverage for plan path detection, legacy-path rejection in
all three tools, inline prompt rendering, and fallback behavior.
* Removes experiment `web-push`.
* Falls back to NoopWebpusher in case of error
* Checks browser capability in FE
* Adds note to agents getting-started docs regarding webpush without TLS
> 🤖
## What
The `lint-actions` CI job only ran when `.github/workflows/ci.yaml` or
`.github/actions/**` changed. New workflow files like `backport.yaml`
and `cherry-pick.yaml` were never linted by zizmor, allowing several
findings to land undetected.
## Changes
**`.github/workflows/ci.yaml`** — Broaden the `ci` path filter from
`".github/workflows/ci.yaml"` to `".github/workflows/**"` so
`lint-actions` runs when any workflow file changes.
**`.github/workflows/backport.yaml`**:
- Move permissions from workflow-level to job-level (`detect` →
`contents: read`, `backport` → `contents: write` + `pull-requests:
write`) — fixes `excessive-permissions`
- Replace `${{ matrix.branch }}` in `run:` block with `$BRANCH` env var
— fixes `template-injection`
- Add `persist-credentials: false` to both checkouts — fixes
`artipacked`
**`.github/workflows/cherry-pick.yaml`** — Add `persist-credentials:
false` to checkout — fixes `artipacked`
**`.github/zizmor.yml`** — Ignore `dangerous-triggers` for
`backport.yaml` and `cherry-pick.yaml`. Both use `pull_request_target`
intentionally — they only run post-merge (`merged == true`) and don't
check out or execute untrusted PR code.
