## Summary
Scope chat model configurations and runtime model resolution to
organizations. Each model configuration now belongs to exactly one
organization, each organization can have its own default model, and
Chatd only uses models that belong to the chat's organization.
Depends on #27968.
## Problem
Chat model configurations are currently deployment-wide even though
chats belong to organizations. A default model, explicitly selected
model, or configured model override can therefore reference the same
model configuration from every organization. This prevents model
catalogs and defaults from being isolated by organization and allows a
chat to retain or select a model that its organization does not own.
Existing model configurations also have no organization ownership or
organization-scoped ACL data, and the database enforces a single live
default across the entire deployment.
## Fix
Add required organization ownership and ACL storage to chat model
configurations. The migration assigns every existing configuration to
the default organization, seeds organization-wide read access, and
replaces the deployment-wide default constraint with a per-organization
constraint.
Scope Chatd lookup, admission, fallback, and queue-promotion behavior to
the chat's organization:
- New chats use the usable default model from their organization.
- Explicit model selections from another organization are rejected.
- Deployment or personal overrides that point to a foreign model are not
used for the chat.
- Historical cross-organization model references remain unchanged in the
database, but runtime generation falls back to a usable local default.
- Queue promotion resolves unavailable, disabled, missing, or foreign
stored models to the usable local default and fails when no usable local
default exists.
The existing model configuration management and availability routes
remain compatibility routes for the default organization in this layer.
A later layer in the stack adds organization-scoped management routes
and updates clients to use them.
## Breaking change
This changes chat model behavior from deployment-wide to
organization-scoped. Existing configurations are migrated only to the
default organization; they are not copied into every organization.
As a result:
- A non-default organization no longer inherits the deployment's
existing default model.
- Creating a chat in an organization without a usable local default
fails with `400: No default chat model config is configured.`
- Explicitly selecting a configuration owned by another organization
fails with `400: Model config does not belong to the specified
organization.`
- Existing chats whose stored model belongs to another organization can
generate only if a usable local default is available; otherwise
generation fails.
Deployments using Agents in non-default organizations must configure a
usable default chat model separately for each organization after the
organization-scoped management layers in this stack are available.
_This pull request description was generated by Coder Agents._
Centralizes every LLM client build in chatd behind one resolver and
makes all model calls behave the same, instead of preserving per-flow
quirks.
## Why
`chatd` built LLM clients in five places (standard turn, compaction
override, quickgen, title override, advisor), each repeating config
parsing, route resolution, client construction, and debug wrapping with
slightly different behavior. An earlier iteration of this PR centralized
construction but preserved every historical quirk behind
purpose-specific spec constructors and policy flags. Review concluded
the quirks themselves were the problem.
## What changed
- `Server.resolveModelCall` (`modelcall.go`) is the single pipeline from
a `modelCallSpec` to a ready client plus call metadata. Callers build
spec literals inline; the only spec fields are data: model source (chat
config, explicit config row, or fixed provider/model pair for computer
use), requested reasoning effort, chatd-scoped routing, and build
options.
- Uniform behavior for every call:
- Provider options are always derived. Whole-chat summaries, turn status
labels, and chat-model compaction summaries previously sent none; they
now follow the config like everything else.
- `MaxOutputTokens` always defaults to 32k when the config leaves it
unset. The compaction summary call nulls the cap after copying the
template because it is non-streaming and the Anthropic SDK rejects
non-streaming requests whose cap implies a completion over ten minutes.
- Debug recording always wraps the client when chat debug is enabled,
labeled with the resolved provider/model. The debug-rebuild paths
(`manualTitleDebugSpec`, `quickgenDebugSpec`) are deleted; debug helpers
now only create and finalize run records, since run association flows
through the generate context, not the model.
- Clients are always constructed with the configured model string and
unmutated route, so AI Gateway validation sees the name exactly as
configured. The resolved identity feeds metadata, logs, and debug labels
only.
- `resolvedModelCall.newCall` / `newObjectCall` are the only production
constructors of `fantasy.Call` / `fantasy.ObjectCall`; object calls now
carry the config's sampling parameters.
- Net: -240 lines.
## Validation
- Full `./coderd/x/chatd/...` suite green (PostgreSQL-backed).
- Red-green toggles verified for the three key unifications (option
derivation, token default, summary cap removal).
- ARCHITECTURE.md: prose the refactor made stale is deleted; no new
resolver documentation is added.
> Mux acted on Mike's behalf for this PR.
Stacked on #27704.
Provider option conversion and reasoning effort injection both create
OpenAI option structs, so each of the four call sites had to pair them
in the right order and pick the same transport for each.
`chatprovider.ProviderOptionsForCall` now owns both steps, and the two
helpers it wraps are unexported. The advisor, main generation,
compaction override, and quickgen paths each collapse to one call.
The ARCHITECTURE section on transport selection is updated to match:
`ProviderOptionsForCall` is described as the only entry point in
`chatprovider` that builds provider options for a call, delegating
OpenAI conversion to `chatopenai.ProviderOptionsFromChatConfig`, and it
now records that the quickgen turn status label and chat summary paths
deliberately send no provider options.
> Mux prepared this PR on Mike's behalf.
Stacked on #27703.
Provider option conversion, reasoning effort injection, and file part
acceptance each recomputed the OpenAI wire format from `(provider,
modelID, override)`. They now read it from `chatprovider.Model`, so a
decision cannot drift from the client it was built for.
`ProviderOptionsFromChatConfig` takes a `Transport`,
`ApplyReasoningEffort` takes a `Model`, and `AcceptsFilePartMediaType`
becomes a `Model` method. `UsesResponsesAPI` and `UsesResponsesOptions`
are deleted. The override extraction is unexported and reachable only
from `ModelFromConfig`, which now takes the model's
`ChatModelOpenAIConfig` directly, removing the six scattered extractions
at call sites.
That also resolves the computer-use mismatch. The computer-use model is
a hardcoded default with no config row of its own: its client was built
without an override while request preparation applied the chat model's.
Preparation now reads the computer-use model's own transport, so the two
agree without one model's client settings following a different model.
Passing the chat model's `openai_config` into the computer-use client
would have made them agree on the wrong value.
`TestModelTransportConsumersAgree` pins the invariant in one test: the
HTTP path the client actually hits, the concrete provider option struct
type, the type created by reasoning effort, and text/image file
acceptance.
> Mux prepared this PR on Mike's behalf.
Stacked on #27683.
The OpenAI wire format is decided when the client is built, then thrown
away, so downstream sites recompute it from `(provider, modelID,
override)`. Any disagreement fails silently: the SDK type-asserts the
concrete provider options struct and discards every OpenAI option, and
text attachments are dropped because Responses natively accepts only
images and PDFs.
This adds `chatprovider.Model`, which pairs a fantasy client with the
transport resolved from that client's own identity. Its fields are
unexported and only the constructor sets the transport, deriving it from
the client, so no caller can pick a transport that disagrees with the
client it wraps. `chatopenai.Transport`'s zero value is invalid and
panics when read rather than defaulting to a wire format, following the
existing precedent for construction invariants.
`Model` is threaded through construction, the resolve paths, and the
four struct fields that store a model for later request preparation.
Terminal call sites keep taking `fantasy.LanguageModel` and receive
`LanguageModel()`, which avoids a new package edge from `chatloop` and
`chatadvisor` into `chatprovider`.
No decisions move yet. The consumers still recompute the transport, and
`UsesResponsesAPI` now delegates to `TransportFor` so the two agree by
construction. #27704 makes the consumers read it from the model.
> Mux prepared this PR on Mike's behalf.
chatd hardcoded `WithUseResponsesAPI()`, so the provider SDK's static
known-model list decided whether an OpenAI model spoke the Responses API
or Chat Completions. A model absent from that list silently fell back to
Chat Completions until the fantasy fork was patched.
This exposes the SDK's `WithResponsesAPIFunc` hook as a per-model
setting, `openai_config.use_responses_api`, stored in the existing
`chat_model_configs.options` JSONB. Unset keeps the known-model list,
`true` forces Responses, `false` forces Chat Completions. There is no
migration.
It sits in a new construction-time `openai_config` section rather than
in `provider_options.openai` because it selects the API when the client
is built, while `provider_options` holds per-request parameters. That
placement is also load-bearing: a config setting only this field would
otherwise materialize an OpenAI request-options struct and turn on
provider-side response storage, since `Store` defaults to true there.
Three places independently decided the transport and would silently
disagree with the client actually built:
| Site | Effect when it disagrees |
| --- | --- |
| `ModelFromConfig` | the transport being overridden |
| `AcceptsFilePartMediaType` | text attachments dropped, since Responses
natively accepts only images and PDFs |
| `UsesResponsesOptions` | the SDK type-asserts the concrete options
struct, so every OpenAI option is discarded |
They share one predicate here, `chatopenai.UsesResponsesAPI`, with the
override threaded to each. The rest of the stack removes that threading
by resolving the transport once and carrying it. Compaction overrides
and the quickgen debug model built clients without `ConfigOptions`, so
they now pass it and pick up both this setting and the existing
Anthropic beta headers.
The toggle also makes transport-conditional option handling
admin-switchable, so two hardening changes ride along.
`ServiceTierFromChat` now maps every tier the codersdk enum advertises
(`auto`, `default`, `flex`, `scale`, `priority`); it previously returned
nil for `default` and `scale`, so flipping a model to Responses silently
dropped a configured `service_tier` that the API accepts (fantasy
forwards the value unchanged). And a new
`TestProviderOptionsTransportParity` pins, per `provider_options.openai`
field, which transport honors it, against a table in ARCHITECTURE.md, so
a field honored on one transport and silently ignored on the other fails
the test unless recorded as intentional.
Review rounds also caught two lifecycle gaps around the new field.
`isZeroChatModelCallConfig` now inspects `OpenAIConfig`, so a stored
options blob whose only setting is this toggle survives into GET/list
responses instead of reading as `model_config: null`;
`TestIsZeroChatModelCallConfigCoversEveryField` sets each config field
in isolation and fails if any field is invisible to the zero check. And
the model editor's update path sends an explicit empty `model_config`
when an edit clears the last field, since an omitted property preserves
the stored options server-side; covered by the
`EditClearingLastOptionSendsEmptyConfig` story.
Azure keeps following the known-model list, because the Azure provider
exposes no equivalent hook. The model editor renders Azure with the
OpenAI option schema, so instead of shipping a visible but inert
control, the option schema generator gains a `providers` struct tag that
it emits as `visible_for_providers`. Gating uses the raw provider type
rather than the alias table, so the control appears only for
openai-typed providers. No hand-written frontend field: the editor
renders it from the generated schema.
Closes
https://linear.app/codercom/issue/CODAGT-874/add-completionsresponses-api-toggle-in-model-editor
> Mux prepared this PR on Mike's behalf.
Stacked on #26657 (the persisted whole-chat summary backend). Base
branch is `chat-summary-62j9`; review/merge that first.
Adds a reusable `ChatSummary` component.
The summary text is the persisted whole-chat summary (`chat.summary`)
introduced by #26657. It is generated asynchronously and may be `null`
until the first summary is produced, in which case the popover renders a
muted empty state. Live updates arrive via that PR's
`chat_summary_change` watch event, which is already merged into the chat
caches.
Cost is served by a new per-chat endpoint, `GET
/api/experimental/chats/{chat}/cost`, which rolls up assistant-message
cost across a chat's root and child (subagent) chats and is authorized
like the other `{chat}` routes (read on the chat, 404 otherwise).
Visual and interaction coverage lives in `ChatSummary.stories.tsx` and
`ChatSummaryPopover.stories.tsx` (including populated-summary,
empty-state, and cost-loading cases).
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Adds a persisted whole-chat summary that backs the chat summary popover.
A new nullable `chats.summary` column is populated in the background
after a successful root-chat turn and pushed to clients via a new
`chat_summary_change` watch event (distinct from `summary_change`, which
is bound to `last_turn_summary`), so the popover reads `chat.summary`
straight off the loaded `Chat` with no extra query.
This is the data source for the popover and per-chat cost UI built in
#26649; the popover can consume `chat.summary` once this lands (the
field is nullable, so merge order does not matter).
## How it works
- **Generation** runs in the existing successful-turn finalize hook,
detached from the request so the user's turn is never blocked. A cadence
gate generates the first summary after one completed turn, then
regenerates every three turns, using the `chats.summary_generated_at`
freshness marker. Generation reads compaction-aware history, renders it
to a bounded plain-text transcript (short transcripts are skipped), and
asks for a 1-3 sentence summary via structured output. Failures never
clear an existing summary.
- **Staleness** is guarded by `history_version` (mirroring
`last_turn_summary`), so a background write racing a newer turn loses
while worker lifecycle transitions cannot reject a fresh write.
- **Model selection** uses the chat's configured model.
## Deferred to follow-ups
- **Cost accounting**: the `chat_messages.cost_source` discriminator and
summary/title usage recording were removed from this PR so summary
persistence is not blocked by hidden accounting rows advancing
`history_version`. Title usage recording stays on main's
`InsertChatMessages` path.
- **Model override**: deployment-wide summary generation model selection
is split into #26803; the base feature always uses the chat model.
## Notes
- Migration `000540` adds `chats.summary` and
`chats.summary_generated_at`, and recreates `chats_expanded` to expose
the new columns.
- Root chats only; shared viewers pick up the summary on their next
refetch (live watch events are owner-only).
Refs #26649
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
> Mux is working on behalf of Mike.
## Summary
Add a per-user synthetic API key for chatd AI Gateway attribution. Chatd
resolves the key from the chat owner, extends it before expiry, and
discards the generated bearer token so the key is never a usable
credential.
There is no mapping table. The key is resolved from `api_keys` by a
deterministic token name (`chatd_<owner_id>_session_token`), mirroring
the provisionerd session token model, with three deltas that chatd
needs:
- **Login type guard**: token names are unvalidated user input, so a
user can create a bearer token with the colliding name. The lookup
excludes `login_type = 'token'` rows, so chatd never picks up (or
extends) a real user token. Synthetic keys are minted with the owner's
login type, which is never `token`.
- **In-place expiry extension instead of delete-and-reinsert**: chat
generations have no stop boundary, and an in-flight generation may have
already delegated the current key ID to aibridged. Extending
`expires_at` keeps the key ID stable forever.
- **Advisory-lock mint**: the unique index on token names is partial
(`WHERE login_type = 'token'`), so nothing DB-enforces uniqueness for
synthetic keys. A per-user advisory lock serializes concurrent mints.
Keys carry a minimal scope (`api_key:read`) as defense in depth; the
delegated gateway path never evaluates scopes and the secret is
discarded at mint.
Migration 000544 removes the foreign keys from the legacy message and
queue `api_key_id` columns while chatd continues stamping them for
rolling compatibility. Stale IDs are tolerated because routing uses
`chats.owner_id`. Individual key deletion, delete-all, and password
reset remove the key without changing chat history or queue versions,
and the next lookup remints it. Suspension does not delete the key;
delegated gateway authorization rejects inactive users at request time.
This is the first PR in a three-PR rollout and must be fully deployed
before #27171.
Refs
https://linear.app/codercom/issue/CODAGT-561/maintain-synthetic-api-key-per-user-per-chat
The chatd state machine only recognizes `waiting`, `running`, `error`,
`requires_action`, and `interrupting`. Remove the unused `pending`,
`paused`, and `completed` values from the database enum, backend, SDK,
frontend, generated queries, and API docs.
Migration `000543_chat_status_remove_unused` remaps existing `pending`
rows to `running`, remaps `paused` and `completed` rows to `waiting`,
drops the obsolete `idx_chats_pending` index, and recreates
`chats_expanded` around the enum swap. It also removes the dead
`AcquireChats` query and all remaining query literals for the deleted
statuses.
**NOTE**: The enum swap can break chat queries from older replicas
during a mixed-version rollout because they still reference
`'pending'::chat_status`. Chats are experimental, so this PR accepts
that limited rollout window instead of adding a two-release expand and
contract sequence.
> This PR was authored by Mux (AI agent) on Mike's behalf.
Coder Agents chats could get stuck showing "Thinking" forever when a
title regenerate/propose request ran while a generation was in flight.
Manual title generation recorded token cost by inserting a hidden
assistant message into `chat_messages` and immediately soft-deleting it.
Triggers on that table sync `chats.history_version` to
`snapshot_version`, so this out-of-band write broke the
`history_version` fence of an in-flight generation task, killing it
without a replacement and leaving the chat stuck in `running`.
Remove the accounting path entirely; AI Gateway already records
title-call usage in `aibridge_interceptions`/`aibridge_token_usages`.
The manual title endpoints no longer write to `chat_messages` at all,
and new regression tests assert `history_version` stays untouched. Note
this intentionally drops title-generation cost from chatd's chat-level
cost surfaces; it still counts against the user's AI budget via AI
Gateway.
Closes CODAGT-595
Closes https://linear.app/codercom/issue/CODAGT-268
## Problem
The chat UI collapses large pastes (>=10 lines or >=1000 chars) into a
synthetic `pasted-text-*.txt` attachment. A chat created with only such
an attachment had no title input anywhere: the create path derived
`titleSource` only from text and file-reference parts (so the chat was
named "New Chat"), async auto-titling extracted text the same way and
silently skipped generation, and the manual propose/regenerate paths
returned an empty title for the same reason. The regular prompt path
already inlines these files for the model; only the title paths were
blind.
## Fix
Add a single title-input derivation in `chatprompt` and use it
everywhere:
- `chatprompt.TitleText` joins text and file-reference parts (unchanged
formatting), and falls back to synthetic pasted-text attachment content
(truncated to a 16 KiB title budget) when they yield nothing.
- `chatprompt.SyntheticPasteFileIDs` identifies paste attachments;
`chatprompt.FallbackTitle` consolidates the previously duplicated
`chatTitleFromMessage` / `fallbackChatTitle`.
- Chat creation captures paste blob references while validating file
parts (the file row was already loaded there) and derives `titleSource`
via `TitleText`. Only the create path derives titles; message send and
edit reuse the same validation without copying any blob data.
- `GenerateChatTitleAsync` and the manual propose/regenerate paths
resolve paste content via `titlePasteText`, which only queries when a
visible user message has no other title text, so chats with typed text
never incur a file fetch.
- Title-path paste fetches are bounded: a new
`GetChatFileDataPrefixesByIDs` query returns only a `substr` prefix
(`chatprompt.TitlePasteBytePrefix`, 64 KiB = 4 bytes x the 16 Ki-rune
title budget) so full blobs (up to 10 MiB each) never leave the database
for titling, and `chatprompt.TitlePasteText` applies the same bound to
the create path which already holds the loaded row.
Deliberate side effect: because generation-time extraction now matches
create-time `titleSource` exactly, file-reference-only chats also become
eligible for AI titles. They were previously skipped by the same
derivation mismatch.
Non-goals: no frontend changes (attachment chip UX stays as is), and
non-synthetic user-uploaded `.txt` files still yield "New Chat".
## Testing
- Unit tests for `TitleText`, `TitlePasteText`, `SyntheticPasteFileIDs`,
`FallbackTitle`, `titleInput`, `titlePasteText`, and paste-aware
`extractManualTitleTurns`.
- Real-database test for `GetChatFileDataPrefixesByIDs` (prefix shorter
and longer than stored data) plus dbauthz coverage for the new query.
- Integration tests: paste-only create gets a fallback title from the
paste content, async title generation fires with the paste content as
input, and `RegenerateChatTitle` works on a paste-only chat.
> This PR was written by [Mux](https://mux.coder.com) on Mike's behalf.
This PR removes the now-dead direct-routing code:
- Deletes the direct routing implementation.
- Collapses the resolvedModelRoute discriminated union into aiGatewayModelRoute.
- Removes the dead providerKeys cascade.
- Deletes the preferredShortTextCandidates quickgen function.
- Simplifies the advisor override error handling.
- Deprecates the AIGatewayRoutingEnabled deployment option. It is now a no-op so as to not break existing deployments on upgrade.
Once direct routing was gone, the AI Gateway became mandatory for chat, which surfaced gaps in how the product behaves with the gateway disabled:
- Exposes ai-gateway-enabled to the frontend via embedded page metadata.
- Disables the chat composer via the existing AgentSetupNotice when the gateway is disabled, for both new and existing chats.
- Fixes nil/typed-nil chatDaemon panics on startup and shutdown when gateway is disabled.
- Fixes chat WebSocket from retrying the still-gated stream endpoint forever when the gateway is disabled.
The provider type already lives authoritatively in ai_providers.type,
reachable on every active row through ai_provider_id, which the
chat_model_configs_ai_provider_required_when_active CHECK makes
mandatory. The stored provider string was a denormalized copy the system
kept in sync with a startup backfill and no longer needs.
Every surface now derives provider type from the linked ai_providers
row. Telemetry is the one exception: it keeps emitting provider, now
sourced from ai_providers.type via a JOIN, so the BigQuery column and the
Nexus dashboards that read it are unaffected. The experimental HTTP/SDK
response drops provider and makes ai_provider_id required, since those
endpoints return only active configs; consumers resolve provider type
from ai_provider_id and the AI providers listing.
This ships in a single release with no compatibility window: production
reads the table via SELECT *, so a pre-drop binary fails config reads the
moment the column is gone. Operators must scale to zero before upgrading,
and there is no rollback.
Closes CODAGT-599
fix(coderd/x/chatd): bind goInflight contexts to server lifetime
- Add Server.inflightContext: WithoutCancel(reqCtx) bound to p.ctx via
context.AfterFunc, so Close cancels in-flight work instead of blocking
on the caller's timeout while a provider is unreachable.
- Apply at GenerateChatTitleAsync, finalizeSuccessfulTurnStatusLabelWithAfterFunc,
setLastTurnSummaryAsync, clearLastTurnSummaryAsync, and scheduleDebugCleanup.
- Honor cleanupCtx in the debug retry-delay timer so cancellation lands
promptly between attempts.
> 🤖
Closes CODAGT-572
## Overview
Bumps `charm.land/fantasy` to the head of `coder_2_33`
(`v0.0.0-20260617050554-2e3ddbca75dd`) and adapts `chatd` to it.
The fantasy bump:
- Syncs upstream `charmbracelet/fantasy` main (v0.31.0) into
`coder_2_33` (coder/fantasy#42).
- Mirrors the request region when prefixing cross-region inference
profiles, so a legacy (un-qualified) Bedrock model ID is prefixed for
the same region the request is actually signed for.
Pulling in the new fantasy version propagates its required transitive
dependency upgrades (aws-sdk-go-v2, OpenTelemetry, google genai,
`golang.org/x/*`, etc.) through MVS, which accounts for the bulk of the
`go.mod`/`go.sum` churn.
## chatd changes
- Thread a per-provider `Region` through `ConfiguredProvider` and
`ProviderAPIKeys` (`RegionByProvider`), and merge/prune/resolve it
alongside API keys and base URLs.
- Source the Bedrock region from AI provider settings in `chatd` and
pass `fantasybedrock.WithRegion` when a region is configured.
- Migrate the runtime Bedrock title-generation model ID to a
fully-qualified `global.anthropic.*` ID.
- Emit a `finish_reason` in the test OpenAI streaming server so streams
close on a terminal event, matching fantasy's fail-closed stream
handling.
## Heads-up: most of this is short-lived
Almost all of the `chatd` code in this PR only executes on the **direct
(non-gateway) routing path** — the branch taken when
`AIGatewayRoutingEnabled` is `false`. That flag was a transition crutch
for AI Gateway routing, and it (plus the entire direct path /
`x/chatd/chatprovider` package that backs it) is slated for removal in
CODAGT-598. Under AI Gateway routing — which is the path every
deployment is expected to run — the Bedrock region is resolved by
aibridge directly from provider settings (`cli/aibridged.go` builds
`aibridge.AWSBedrockConfig{Region: settings.Bedrock.Region}`), so none
of the region plumbing added here is reached.
Concretely, expect the following to be deleted alongside the direct
path:
- The `RegionByProvider` map, the `Region()` accessor, and the region
preservation in merge/resolve plus the region pruning in
`PruneDisabledProviderKeys` (`chatprovider.go`).
- The `fantasybedrock.WithRegion(region)` branch in `ModelFromConfig` —
only reachable on the direct path; the gateway path builds a
`fantasyanthropic` client with no region key.
- Reading `settings.Bedrock.Region` in `aiProviderConfigFromKeys`
(`chatd.go`).
- The region-specific tests in `chatprovider_test.go`, and the
`chattest`/`model_coverage` adjustments that support direct-path
testing.
What survives the cleanup (independent of routing):
- The `charm.land/fantasy` bump and its `go.mod`/`go.sum` transitive
churn.
- The fully-qualified `global.anthropic.*` Bedrock title-generation
model ID in `quickgen.go` (a runtime-valid model identifier, not
direct-path-specific).
We're landing the full change anyway so the direct path stays correct
for the remaining transition window; just don't be surprised when
CODAGT-598 reclaims most of it.
## Notes
Depends on coder/fantasy `coder_2_33` already containing the upstream
sync and Bedrock region fix (merged via coder/fantasy#42 and
coder/fantasy#43).
Using `WaitGroup.Go` must be synchronized with `WaitGroup.Wait`
according to [go docs](https://pkg.go.dev/sync#WaitGroup.Go):
> If the WaitGroup is empty, Go must happen before a
[WaitGroup.Wait](https://pkg.go.dev/sync#WaitGroup.Wait).
There were a couple of places in chatd that violated this principle.
This was caught as a data race in
https://github.com/coder/internal/issues/1599. This PR ensures that all
functions that spawn inflight goroutines synchronize with each other.
I also noticed that inflight goroutines may be spawned after the server
is closed, which was surprising and looked like a bug. This PR therefore
also introduces a mechanism that disallows spawning inflight goroutines
after the server is closed, and ensures that any code that tries doing
it logs an error.
Closes https://github.com/coder/internal/issues/1599.
## Summary
Routes chatd model calls backed by concrete AI Provider rows through the
in-process aibridge transport by default, with deployment options to use
direct provider routing when AI Gateway is disabled or chat AI Gateway
routing is disabled.
- Splits model routing into common, direct provider, and AI Gateway
paths behind a single deployment-mode entry point.
- Builds chatd models through explicit request, route, and options data.
Active API key attribution is passed explicitly instead of being hidden
inside generic model construction.
- For AI Gateway BYOK routes, resolves the user's provider key in chatd,
forwards it through provider-specific auth headers, and sets
`X-Coder-AI-Governance-Token` to the `delegated` marker so aibridge
preserves those headers while still stripping Coder-specific metadata.
- Keeps central provider credentials and deployment fallback credentials
out of forwarded provider auth headers, so AI Gateway central policy
remains authoritative.
- Redacts delegated provider auth from default string formatting to
avoid accidental plaintext logging of user BYOK credentials.
- Covers selected chat models, advisor overrides, title and quickgen
paths, subagent overrides, computer use model selection, and an
integration-style chat turn through the aibridge transport path.
- Persists initiating API key IDs on chat and queued user messages,
including subagent child messages, and fails closed for AI
Gateway-routed model builds without an active key.
- Removes unused `api_key_id` indexes while keeping the persistence
columns and foreign keys.
- Keeps the deployment option available through config and env parsing,
but hides it from CLI help and generated docs.
- Stabilizes the subagent poll fallback test so background CreateChat
processing cannot win the state transition under slower CI environments.
## Tests
- `go test ./coderd/x/chatd -run
'TestAIGatewayProviderAuthForUser|TestAIGatewayProviderAuthRedactsFormatting|TestResolveModelRouteForConfigAIGatewayProviderAuth|TestAIGatewayModelForwardsProviderAuth|TestProcessChat_AIGatewayRoutingUsesDelegatedAPIKey|TestAwaitSubagentCompletion'
-count=1`
- `go test ./coderd/aibridged -run
'TestServeHTTP_DelegatedAPIKey|TestServeHTTP_StripCoderToken' -count=1`
- `git diff --check HEAD~1..HEAD`
- `make lint`
> Mux working on behalf of Mike.
> Mux is acting on Mike's behalf.
Changes chat turn-end summaries into compact status labels for the
cached `last_turn_summary` and successful web push body.
Uses a structured-output model call for successful turns, requiring a
2-5 word `label` and validating it to reject agent-centric phrasing.
Pending and requires-action states keep deterministic status labels.
Removes the earlier deterministic tool-signal pipeline in favor of the
smaller structured-output path.
The async title-generation and turn-summary goroutines launched from
processChat run autocommit UPDATEs on the chat row after finishActiveChat
has set the chat to pending and signalWake has fired. If the row lock
from one of those UPDATEs is held while acquireLoop's processOnce runs,
AcquireChats's FOR UPDATE SKIP LOCKED skips the freshly-pending chat and
returns no rows. The wake is then consumed with no acquisition, and the
chat sits in pending until the next acquireTicker (default 1s).
Wake again after each UPDATE commits. The second wake covers the race
window without changing the transaction semantics.
Closescoder/internal#1500
Persists the agent-generated turn-end summary on `chats` and shows it as
the Agents sidebar subtitle when present, falling back to the model
name. Errors still take precedence.
> Mux is acting on Mike's behalf.
## What changes
**Storage.** New nullable `last_turn_summary` column on `chats`
(migration `000486`). New `UpdateChatLastTurnSummary` query normalizes
blank/whitespace input to `NULL`, preserves `updated_at` (so the chat
does not jump to the top of the sidebar on summary writes), and uses an
`expected_updated_at` stale-write guard so an older async summary cannot
overwrite a newer turn.
**Backend.** `coderd/x/chatd/chatd.go` decouples summary generation from
webpush. Generated summaries persist for completed parent turns even
when webpush is unconfigured or has no subscriptions. The same generated
text is reused as the webpush body when webpush is configured, so the
summary model is not called twice. Generic fallback push text is no
longer persisted; it clears any stale summary instead.
Error/interrupt/pending-action terminal paths clear `last_turn_summary`
for the latest turn.
**Frontend.** `AgentsSidebar.tsx` subtitle priority is now `errorReason
|| lastTurnSummary || modelName`, normalized via the existing
`asNonEmptyString` helper from `blockUtils.ts`.
## Tests
- `TestUpdateChatLastTurnSummary` (database): success,
whitespace-to-NULL, stale guard rejects, `updated_at` preserved.
- `TestUpdateLastTurnSummaryRejectsStaleWrites` (chatd internal): direct
stale-`expected_updated_at` test.
- `TestSuccessfulChatPersistsTurnSummaryWithoutWebPush`: persistence
works without webpush subscriptions.
- `TestSuccessfulChatSendsWebPushWithSummary`: same generated text
drives both DB and push body.
-
`TestSuccessfulChatSendsWebPushFallbackWithoutSummaryForEmptyAssistantText`:
fallback text is not persisted.
- `TestErroredChatClearsLastTurnSummaryAndSendsWebPush`: error path
clears the field.
- `TestInterruptChatDoesNotSendWebPushNotification`: interrupt path
clears the field, no push fires.
- `AgentsSidebar.test.tsx`: subtitle priority for summary-present,
error-wins, no-summary fallback, whitespace fallback.
- `AgentsSidebar.stories.tsx`: `ChatWithTurnSummary` and
`ChatWithTurnSummaryAndError`.
## Notes
- No backfill. Existing chats keep showing the model name until their
next turn completes.
- Parent chats only in this iteration; the field is rendered on any
`Chat` if a future change extends generation to children.
- Decoupling generation from webpush adds quickgen model calls for
completed parent turns that previously skipped generation when no
subscriptions existed. Existing parent-only, assistant-text-present,
`PushSummaryModel` configured, and bounded-timeout gates keep this
behavior bounded.
Adds an admin-configurable deployment-wide setting that controls which
model is used for chat title generation. Admins can pick any enabled
chat model config from the Agents settings page, or leave the setting
unset to keep the existing fast-models-then-chat-model fallback
algorithm.
When a model is selected, both automatic and manual title generation use
only that model, with no silent fallback. When the configured model is
disabled, missing credentials, or otherwise unusable, automatic title
generation skips entirely (best-effort) and manual title regeneration
returns a clear error, so admins notice the misconfiguration instead of
silently routing title traffic through another provider.
## Surface
- New deployment-wide setting stored as a `site_configs` row
(`agents_chat_title_generation_model_override`).
- New experimental endpoint `GET/PUT
/api/experimental/chats/config/model-override/{context}`.
- Frontend: title generation now appears as a third dropdown on the
Agents admin settings page alongside the existing general and explore
context overrides.
## DRY refactors folded in
Title generation is integrated as a third value of the existing
`ChatModelOverrideContext` type alongside `general` and `explore`,
sharing the parameterized HTTP route, SDK methods, generated types, and
frontend API plumbing rather than introducing a parallel surface. The
`Agent` prefix was dropped from the type and route since title
generation is not a delegated agent.
The chatd model-override resolver is also shared.
`resolveConfiguredModelOverride` now takes a `failureMode` parameter:
- Subagent overrides use soft failure: misconfigured overrides are
logged and the parent model is used.
- Title generation uses hard failure: misconfigured overrides return an
explicit error so manual title regeneration surfaces the
misconfiguration and automatic title generation skips instead of
silently falling back.
> Mux is acting on Mike's behalf.
Chat title generation used free-form text completion, which let models
respond conversationally instead of producing a title. Review chats
started with GitHub URLs were especially affected — models would say "I
don't have the ability to browse external links" and that string became
the persisted title.
Replace the raw-text `generateShortText` path with structured output via
`object.Generate[generatedTitle]`. Both auto-title and manual retitle
now go through the same typed contract: the model must return a JSON
object with a `title` field, validated and normalized before
persistence. Invalid outputs (empty, too long) are rejected and retried
through the existing candidate-model fallback loop.
Short prompts were producing title-generation meta responses such as "I
am a title generator" and prompt-echo titles. This rewrites the
automatic and manual title prompts to be shorter, less self-referential,
and more focused on returning only the title text.
The change also removes the broader post-generation guard layer, updates
manual regeneration to send real conversation text instead of a meta
instruction, and keeps regression coverage focused on the slimmer prompt
contract.
## Summary
Adds a "Generate new title" action that lets users manually regenerate a
chat's title using richer conversation context than the automatic
first-message title path.
## Changes
### Backend
- **New endpoint:** `POST
/api/experimental/chats/{chatID}/title/regenerate` returns the updated
Chat with a regenerated title
- **Manual title algorithm:** Extracts useful user/assistant text turns
→ selects first user turn + last 3 turns → builds context with gap
markers → renders prompt with anti-recency guidance → calls lightweight
model → normalizes output
- **Helpers:** `extractManualTitleTurns`,
`selectManualTitleTurnIndexes`, `buildManualTitleContext`,
`renderManualTitlePrompt`, `generateManualTitle` — all private, with the
public `Server.RegenerateChatTitle` method
- **SDK:** `ExperimentalClient.RegenerateChatTitle(ctx, chatID) (Chat,
error)`
- Persists title via existing `UpdateChatByID` and broadcasts
`ChatEventKindTitleChange`
### Frontend
- API client method + React Query mutation with cache invalidation
- "Generate new title" menu item (with wand icon) in both TopBar and
Sidebar dropdown menus
- Loading/disabled state while regeneration is in-flight
- Error toast on failure
- Stories updated for both menus
### Tests
- `quickgen_test.go`: Table-driven tests for all 4 helper functions
(turn extraction, index selection, context building, prompt rendering)
- `exp_chats_test.go`: Handler tests (ChatNotFound,
NotFoundForDifferentUser, NoDaemon)
## Design notes
- The existing auto-title path (`maybeGenerateChatTitle`, `titleInput`)
is completely unchanged
- Manual regeneration uses richer context (first user turn + last 3
turns + gap markers) vs the auto path's single first message
- Endpoint is experimental and marked with `@x-apidocgen {"skip": true}`
- Add `X-Coder-Owner-Id`, `X-Coder-Chat-Id`, `X-Coder-Subchat-Id`,
`X-Coder-Workspace-Id` headers to all outgoing LLM API requests from
chatd
- Extend `ModelFromConfig` with `extraHeaders` param, forwarded via
Fantasy `WithHeaders` on all 8 providers
- Add `CoderHeaders(database.Chat)` helper to build the header map from
chat state
- Update all 4 `ModelFromConfig` call sites (resolveChatModel,
computer-use override, title gen, push summary)
- Thread `database.Chat` into `generatePushSummary` (was `chatTitle
string`)
- Tests: `TestCoderHeaders` (4 subtests),
`TestModelFromConfig_ExtraHeaders` (OpenAI + Anthropic),
`TestModelFromConfig_NilExtraHeaders`
- Refactor existing `TestModelFromConfig_UserAgent` to use channel-based
signaling
> 🤖 This PR was generated by Coder Agents and self-reviewed by a human.
- Moves `coderd/chatd/`, `coderd/gitsync/`, `enterprise/coderd/chatd/`
under `x/` parent directories to signal instability
- Adds `Experimental:` glue code comments in `coderd/coderd.go`
> 🤖 This PR was created with the help of Coder Agents, and was
reviewed by my human. 🧑💻
Ethan