## Summary
Scope chat model configurations and runtime model resolution to
organizations. Each model configuration now belongs to exactly one
organization, each organization can have its own default model, and
Chatd only uses models that belong to the chat's organization.
Depends on #27968.
## Problem
Chat model configurations are currently deployment-wide even though
chats belong to organizations. A default model, explicitly selected
model, or configured model override can therefore reference the same
model configuration from every organization. This prevents model
catalogs and defaults from being isolated by organization and allows a
chat to retain or select a model that its organization does not own.
Existing model configurations also have no organization ownership or
organization-scoped ACL data, and the database enforces a single live
default across the entire deployment.
## Fix
Add required organization ownership and ACL storage to chat model
configurations. The migration assigns every existing configuration to
the default organization, seeds organization-wide read access, and
replaces the deployment-wide default constraint with a per-organization
constraint.
Scope Chatd lookup, admission, fallback, and queue-promotion behavior to
the chat's organization:
- New chats use the usable default model from their organization.
- Explicit model selections from another organization are rejected.
- Deployment or personal overrides that point to a foreign model are not
used for the chat.
- Historical cross-organization model references remain unchanged in the
database, but runtime generation falls back to a usable local default.
- Queue promotion resolves unavailable, disabled, missing, or foreign
stored models to the usable local default and fails when no usable local
default exists.
The existing model configuration management and availability routes
remain compatibility routes for the default organization in this layer.
A later layer in the stack adds organization-scoped management routes
and updates clients to use them.
## Breaking change
This changes chat model behavior from deployment-wide to
organization-scoped. Existing configurations are migrated only to the
default organization; they are not copied into every organization.
As a result:
- A non-default organization no longer inherits the deployment's
existing default model.
- Creating a chat in an organization without a usable local default
fails with `400: No default chat model config is configured.`
- Explicitly selecting a configuration owned by another organization
fails with `400: Model config does not belong to the specified
organization.`
- Existing chats whose stored model belongs to another organization can
generate only if a usable local default is available; otherwise
generation fails.
Deployments using Agents in non-default organizations must configure a
usable default chat model separately for each organization after the
organization-scoped management layers in this stack are available.
_This pull request description was generated by Coder Agents._
Closes
[ENG-2869](https://linear.app/codercom/issue/ENG-2869/flake-testpostchatmessagesbusyinterrupt).
The test used to assert a transient chat state, so I got rid of that
assertion. There was also a related race in `interruptChat` where the
test pubsub message buffer could be cleared after a runner posted the
pubsub message that tests look for.
Closes CODAGT-835
## Summary
`chat_messages.runtime_ms` becomes the billing source of truth for Coder
Agents runtime (summed hourly by #27312), but it was built for
debugging: the June refactor (#26270) silently stopped recording
tool-step runtime, compaction was never measured, and interrupted turns
lost their partial runtime entirely. This PR defines the billable
metric, closes the paths that dropped it, and documents the definition
where the data lives.
## The billable definition
**`runtime_ms` is the wall-clock duration of the model invocation that
produced the persisted message content**, measured from just before the
provider stream opens until it is fully consumed.
What counts:
- Assistant generation steps, in top-level and sub-agent chats
(sub-agents are ordinary chats on the same generation path).
- Compaction summarization calls, persisted on the compaction assistant
message (**new**).
- Interrupted attempts: the message-part episode's lifetime is persisted
on the partial assistant message committed by `FinishInterruption`, so
partial generation time survives interruption (**new**; measured via a
new `Buffer.EpisodeDuration`, which works even though the generation
goroutine and the interrupt task are different tasks).
What deliberately does not count (each is documented in code and docs):
- **Local tool execution.** Tool wall time includes idle waits, most
importantly `wait_agent` polling a sub-agent chat that already bills its
own model invocations; billing the batch would double count, and
excluding one tool from a concurrent batch's wall time is ill-defined.
Pre-refactor instrumentation did include tool time; this makes the
exclusion an explicit product definition instead of a silent regression.
- **Failed model calls whose output is discarded** (retried attempts,
terminal errors, content-filter refusals). They persist no content, so
they bill nothing; billing errs toward undercounting. Notably a
stream-silence timeout can burn 10 idle minutes before a retry, which
should not be billable "active generation". If product later wants
failed attempts billed, that needs a place to persist runtime on error
turns (`FinishError` inserts no rows today) and is a deliberate
follow-up, not instrumentation drift.
- **Ancillary calls that produce no chat messages** (title generation,
advisor, turn summaries) and all idle/parked time (`requires_action`,
queueing).
The definition is documented as `COMMENT ON COLUMN
chat_messages.runtime_ms` (migration 000551, surfacing as a Go doc
comment on `ChatMessage.RuntimeMs`), on
`chatloop.PersistedStep.Runtime`, in the chatd architecture doc, and in
the Spend Management docs page.
## Index for the hourly scan
None needed: `GetTotalChatMessageRuntimeMsInRange` (#27312) filters an
hour-wide `created_at` range, which the existing
`idx_chat_messages_created_at` b-tree already serves; the residual
`runtime_ms IS NOT NULL` filter applies to one hour of rows. A partial
index would add permanent write amplification for a query that runs once
an hour.
> [!NOTE]
> Migration 000551 is also claimed by #27312; whichever merges second
renumbers via `fix_migration_numbers.sh`.
## Tests
- End-to-end: the existing full-server generation test now asserts
`RuntimeMs.Valid` on the committed assistant row (it previously read
`.Int64` without checking `.Valid`, so it passed on NULL).
- Interrupted turn: full task-level test (real DB, mock clock) asserting
the partial assistant message persists the attempt's runtime.
- Errored stream: asserts a failed invocation yields no step and no
runtime.
- Tool-using turn: asserts runtime lands on the assistant row only and
tool rows stay NULL.
- Compaction: asserts the summarization call duration is recorded and
lands on the compaction assistant message only.
- `messagepartbuffer.EpisodeDuration` unit coverage.
Blocks: CODAGT-843 (B3), CODAGT-838 (D8).
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Hugo Dutka <hugo@coder.com>
> Mux is working on behalf of Mike.
## Summary
Stop reading and writing the legacy `api_key_id` columns on chat
messages and queued messages, and drop the columns in the same PR.
Runtime AI Gateway attribution continues to use the per-user synthetic
key introduced by #27170.
With the columns gone, `sqlc` generates `database.ChatMessage` and
`database.ChatQueuedMessage` without `api_key_id`, so no transitional
query scaffolding is needed.
Migration `000548` drops the `api_key_id` columns. #27170 already
removed their foreign keys, so the down migration re-adds nullable text
columns without constraints. Previous column values cannot be restored.
Also moves the model config validation in `CreateChat` above the
message-building work so a disabled or invalid model fails fast. On main
this mattered more: the old ordering minted a synthetic API key before
rejecting the request.
Deploy note: replicas still running the previous release write
`api_key_id` on insert, so chat message inserts on old replicas fail
during the rolling window after the column drop. This was previously
split across two PRs to avoid that window; per review feedback the split
added more churn than it was worth for an experimental surface.
Depends on #27170 (merged).
The provider type already lives authoritatively in ai_providers.type,
reachable on every active row through ai_provider_id, which the
chat_model_configs_ai_provider_required_when_active CHECK makes
mandatory. The stored provider string was a denormalized copy the system
kept in sync with a startup backfill and no longer needs.
Every surface now derives provider type from the linked ai_providers
row. Telemetry is the one exception: it keeps emitting provider, now
sourced from ai_providers.type via a JOIN, so the BigQuery column and the
Nexus dashboards that read it are unaffected. The experimental HTTP/SDK
response drops provider and makes ai_provider_id required, since those
endpoints return only active configs; consumers resolve provider type
from ai_provider_id and the AI providers listing.
This ships in a single release with no compatibility window: production
reads the table via SELECT *, so a pre-drop binary fails config reads the
moment the column is gone. Operators must scale to zero before upgrading,
and there is no rollback.
Closes CODAGT-599
This PR adds logging when the chat runner retries and exits because of
an error. It also adds a 15-minute task timeout to ensure that stuck
tasks do not hang forever.
Ethan