mirror of
https://github.com/coder/coder.git
synced 2026-11-01 04:47:17 +08:00
d014d4c372fb7dfdb3b0ea00a9935fec9b9528f9
9
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
4b7494be72 |
feat: harden chat generation runtime instrumentation for billing (#27451)
Closes CODAGT-835 ## Summary `chat_messages.runtime_ms` becomes the billing source of truth for Coder Agents runtime (summed hourly by #27312), but it was built for debugging: the June refactor (#26270) silently stopped recording tool-step runtime, compaction was never measured, and interrupted turns lost their partial runtime entirely. This PR defines the billable metric, closes the paths that dropped it, and documents the definition where the data lives. ## The billable definition **`runtime_ms` is the wall-clock duration of the model invocation that produced the persisted message content**, measured from just before the provider stream opens until it is fully consumed. What counts: - Assistant generation steps, in top-level and sub-agent chats (sub-agents are ordinary chats on the same generation path). - Compaction summarization calls, persisted on the compaction assistant message (**new**). - Interrupted attempts: the message-part episode's lifetime is persisted on the partial assistant message committed by `FinishInterruption`, so partial generation time survives interruption (**new**; measured via a new `Buffer.EpisodeDuration`, which works even though the generation goroutine and the interrupt task are different tasks). What deliberately does not count (each is documented in code and docs): - **Local tool execution.** Tool wall time includes idle waits, most importantly `wait_agent` polling a sub-agent chat that already bills its own model invocations; billing the batch would double count, and excluding one tool from a concurrent batch's wall time is ill-defined. Pre-refactor instrumentation did include tool time; this makes the exclusion an explicit product definition instead of a silent regression. - **Failed model calls whose output is discarded** (retried attempts, terminal errors, content-filter refusals). They persist no content, so they bill nothing; billing errs toward undercounting. Notably a stream-silence timeout can burn 10 idle minutes before a retry, which should not be billable "active generation". If product later wants failed attempts billed, that needs a place to persist runtime on error turns (`FinishError` inserts no rows today) and is a deliberate follow-up, not instrumentation drift. - **Ancillary calls that produce no chat messages** (title generation, advisor, turn summaries) and all idle/parked time (`requires_action`, queueing). The definition is documented as `COMMENT ON COLUMN chat_messages.runtime_ms` (migration 000551, surfacing as a Go doc comment on `ChatMessage.RuntimeMs`), on `chatloop.PersistedStep.Runtime`, in the chatd architecture doc, and in the Spend Management docs page. ## Index for the hourly scan None needed: `GetTotalChatMessageRuntimeMsInRange` (#27312) filters an hour-wide `created_at` range, which the existing `idx_chat_messages_created_at` b-tree already serves; the residual `runtime_ms IS NOT NULL` filter applies to one hour of rows. A partial index would add permanent write amplification for a query that runs once an hour. > [!NOTE] > Migration 000551 is also claimed by #27312; whichever merges second renumbers via `fix_migration_numbers.sh`. ## Tests - End-to-end: the existing full-server generation test now asserts `RuntimeMs.Valid` on the committed assistant row (it previously read `.Int64` without checking `.Valid`, so it passed on NULL). - Interrupted turn: full task-level test (real DB, mock clock) asserting the partial assistant message persists the attempt's runtime. - Errored stream: asserts a failed invocation yields no step and no runtime. - Tool-using turn: asserts runtime lands on the assistant row only and tool rows stay NULL. - Compaction: asserts the summarization call duration is recorded and lands on the compaction assistant message only. - `messagepartbuffer.EpisodeDuration` unit coverage. Blocks: CODAGT-843 (B3), CODAGT-838 (D8). --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Hugo Dutka <hugo@coder.com> |
||
|
Michael Suchacz
|
6b8f820493 |
feat: remove native chat cost tracking in favor of AI Gateway cost data (#27330)
## Stack Context This stack makes AI Gateway data and budgets the source of truth for AI spend controls. 1. Re-back the per-chat cost endpoint with AI Gateway data (#27328, merged). 2. Remove native chat usage limits (#27329, merged). 3. **This PR, now based on `main`:** remove native chat cost tracking and its dedicated admin UI. ## Summary Removes native per-message price calculation, model pricing fields, cost persistence, aggregate cost queries, and admin cost API types. It also deletes the Analytics and Spend pages plus their legacy redirects. The AI Gateway-backed per-chat cost row and compact budget indicators remain. The spend documentation is renamed to `spend-management.md` and updated for the remaining surfaces, group budget APIs, CSV export, upgrade handling for native pricing and cost history, and the absence of a deployment-wide spend dashboard. The per-chat cost API documents that data follows AI Gateway retention and reports zero after all matching requests are purged. No schema is dropped in this release. `chat_messages.total_cost_micros` remains nullable and unwritten so replicas from the previous release can continue inserting messages during rolling upgrades. #27600 tracks removal after the compatibility window. > Mux prepared this PR on Mike's behalf. |
||
|
Michael Suchacz
|
df1c0f9710 |
feat: show what a chat lifecycle hook changed (#27655)
## Stack Context Follow-up fixes from live UAT of the merged chat lifecycle hooks stack (#27430). Two PRs: 1. **This PR**: make hook effects visible and correctly attributed in the transcript. 2. [`mike/chat-hooks-uat/dispatch-capacity`]: reserve dispatch capacity so an admission burst can't fail running turns. ## Why? UAT found three ways the transcript misrepresented what a lifecycle hook did. All three are user-visible and share the same surface (`chathooks/effects.go`, `codersdk.ChatMessagePart`, the conversation timeline), so they're reviewed together. **A prompt `input_override` silently discarded attachments.** `ComposeUserPromptContent` replaced the entire submitted part list with one text part, dropping `file` and `file-reference` parts along with their `chat_file_links`. The user saw their attachments vanish with no explanation. The override now replaces submitted *text* parts only and preserves non-text parts in order. A consumer that wants to block attachments uses `deny`, which is the documented mechanism for refusing a submission. **Every user-visible `system` row was labelled "Lifecycle hook".** The timeline keyed the notice off `role === "system"`. That was correct only by accident, because the hook `user_message` was the sole client-visible system row. The backend now emits the notice as a typed `hook-notice` part and the timeline renders on that, so a future system row can't be mislabelled as a policy notice. **Nothing marked a tool call the hook had rewritten.** A consumer could replace tool input via `input_override` and the transcript showed the rewritten input as if the model had produced it. `ChatMessagePart` gains `hook_rewritten`, set from `preflight.Overrides` on the same path that already carries `ToolCallCreatedAt`, and the tool row renders a "Modified by policy" badge. `ToolCall.PolicyProvider` renders the badge itself, at four wrap sites: the `Tool` dispatch wrapper, the `ReadFilesTool` aggregate and its per-file rows, and `ReadFileTimelineBlock` (grouped and single `read_file` rows bypass `Tool`). Renderer props do not include the flag; descendants consume it through the provider context. The badge is emitted by the provider rather than by the shared header because several renderer branches return early without one, including the auth-required `execute` card, a completed `ask_user_question`, and an empty question payload. Those branches would drop the attribution with no type or runtime error, and the gap is not greppable: every renderer file contains a header somewhere, only individual branches do not. Emitting at the provider removes the possibility instead of enumerating the cases. A rewritten call is wrapped in a group labelled by its badge, so one rewritten file inside a merged read is attributed on its own rather than inheriting the group's badge. `HeaderButton` still appends the policy wording to an explicit `ariaLabel`, since an explicit `aria-label` replaces the name computed from descendants. Provider-executed calls are excluded from attribution. Hooks never see them, and duplicate tool-call ID rejection deliberately skips them, so a reused ID would otherwise mark a provider-executed call as policy-rewritten. ## Testing Go: `coderd/x/chatd/...`, `coderd/x/agenthooks/...`, `codersdk/...`, and `coderd -run 'Hook|Chat'`. Frontend: `tsc` plus every `AgentsPage` story; the only failures are `MCP Tool Completed` and `Scroll To Bottom Button Works With Inverse Scroll`, both of which fail on trunk. A registry-wide story asserts every registered renderer shows the badge, verified against three inverted toggles: removing the badge, hiding it with `display:none`, and skipping the provider for one renderer (which names that renderer). Storybook also covers the rewritten subagent spawn, a completed empty question payload, a non-hook system message, and a failed `read_file` guarding the accessible name. > Mux opened this PR on Mike's behalf. |
||
|
Michael Suchacz
|
c17bed25e0 |
feat: wire chat lifecycle hooks into chatd (#27429)
Wires chat lifecycle hooks into chatd, gated by the `agent-lifecycle-hooks` experiment. Part of the lifecycle hooks stack (#27401, #27428, #27430). See `docs/admin/setup/chat-lifecycle-hooks.md` for the consumer-facing contract. ## Summary When a hook URL is configured, chatd dispatches `session_start`, `user_prompt_submit`, `pre_tool_use`, `post_tool_use`, `pre_compact`, `post_compact`, and `stop` events to the consumer and applies its responses. ## Design - **Stateless**: Coder stores no hook dispatch or decision state. Delivery is at least once; consumers deduplicate on stable payload identifiers (chat ID, event type, tool-use ID) and answer duplicates with the same decision. - **Admission-time prompt effects**: `user_prompt_submit` dispatches exactly once per submission (create, send, queue, edit, subagent spawn) and folds its effects into the stored prompt as typed message parts: original-or-overridden user parts, then model-only `hook-context`, then a user-visible `hook-notice`. Hook context is stripped from every client-facing conversion; hook notices are excluded from model prompts. The server rejects hook parts in client-submitted content. - **Tool gating**: `pre_tool_use` allow can override tool input; deny becomes a synthetic denied tool result, with any returned model context persisted as a model-only transcript row so it never reaches clients. The denial text identifies an external policy (the deployment's lifecycle hook) as the source and marks the decision as persistent, so the model explains the denial instead of retrying it or misreporting it as an infrastructure failure. - **Fail closed**: a dispatch failure rejects the triggering request or moves the chat to the error state in the same transaction as the affected step, so a runnable state is never published with unapproved content. - **Admission before persistence**: `pre_tool_use` is dispatched for the calls the model produced, before the assistant message is stored. See "Staged tool admission" below. - **Fresh dispatch per tool call**: every non-provider-executed tool call is decided by its own `pre_tool_use` dispatch; Coder never reuses an earlier decision on the consumer's behalf. Retries re-dispatch the same logical event. ## Structure All hook dispatch flows through one seam: entry points build a `chathooks.Chat` (chat identity) and a `chathooks.Message` (event details) and call `Trigger.Trigger`, the only component that talks to the dispatcher. The integration lives in the `coderd/x/chatd/chathooks` subpackage, split by responsibility: - `trigger.go`: the trigger seam; builds the wire envelope per event, normalizes deny into a typed error, and holds the package's single enabled-check. - `effects.go`: pure conversion of hook results into transcript rows and prompt parts. - `errors.go`: failure classification (dispatch error messages, denial mapping, tool-result dispatch-failure scanning). - `tooluse.go`: the tool-call gate (`pre_tool_use` preflight, `post_tool_use` payloads, applying admitted input to the step). Server-bound glue stays in `coderd/x/chatd/hook_server.go`: the chat-parking dispatch error handlers, the step-commit row insertion wrappers, and the dynamic post-tool-use state loader, which depends on chatd validation types. This PR adopts the `codersdk/x/agenthooks` and `coderd/x/agenthooks/dispatch` import paths introduced at the tip of #27401; intermediate commits still reference the pre-move paths and are not individually buildable. ## Staged tool admission `pre_tool_use` originally ran at tool execution time, which is after the assistant message carrying the tool call was already committed. An `input_override` therefore had to rewrite stored message content in place. @hugodutka pointed out that chatd treats message content as immutable, and that the rewrite was a shortcut rather than a requirement. It was also a correctness problem in its own right: the rewrite only updated the database, so the transcript could show one input while a different one had executed. The hook now runs before the step is persisted: ```text provider stream ends (tool calls complete, in memory) -> pre_tool_use dispatch per call -> ONE transaction: assistant row with admitted inputs, synthetic denials, hook rows -> execute ``` The step is inserted once, carrying the input the tool runs with. `UpdateChatMessageContentByID` and `Tx.UpdateMessageContent` are deleted from #27428, so message content stays immutable. Two consequences, both intentional: - **Clients converge rather than wait.** Tool-call parts still stream live, so a rewritten call briefly shows the model's proposed input before the committed message replaces it. The chat store already clears stream state when an assistant message arrives, so the stored input wins with no frontend change and no added latency before tool cards appear. - **A call already in history was already admitted.** Execution consumes the stored input instead of dispatching a second decision, which keeps one dispatch and one set of hook effects per call. A consumer policy change between admission and execution applies to later calls, not to calls already admitted. The per-chat debug endpoint still records the provider's original tool input. Its purpose is to report provider behavior, and it requires an explicit per-chat debug flag; the invariant here covers the transcript. ## Configuration Adds `chat-hook-url`, `chat-hook-secret`, `chat-hook-timeout`, and `chat-hook-enabled` deployment options with startup validation. The flags are hidden from `coder server --help` while the feature is experimental; the setup guide documents them. ## Tool input validation Built-in tool arguments reach a consumer as raw JSON with key spelling preserved, but the tools decode those bytes with Go, which matches struct fields case-insensitively and keeps the last match. A policy reading `path` could therefore authorize one value while the tool executed another, and a lone case variant such as `{"PATH":"/secret"}` was invisible to a policy checking for `path`. Coder now rejects a built-in tool call whose input repeats a key or spells a schema property with different capitalization, before the `pre_tool_use` dispatch, so a consumer is never asked to authorize bytes whose meaning depends on the reader. Rejected calls produce an error result the model can retry; unambiguous calls in the same batch still run. A consumer-authored `input_override` is rechecked after the dispatch and fails the turn closed, because the model cannot correct it. Dynamic and MCP inputs are excluded because the client and the workspace agent execute those calls rather than coderd. Two paths needed more than a schema check. Execution resolves a deprecated tool name to its canonical tool, so validation resolves aliases first. The `edit_files` decoder also reads `search` and `replace`, which its schema does not advertise, so those aliases are now matched exactly and their case variants ignored. A hook denial now returns a structured 403 carrying `kind: "hook_denied"`, mirroring the dispatch-failure response that already carries its own kind. Without it a client cannot tell a policy decision apart from a generic failure, and the chat UI titled a denial "Request failed". Adding a kind needs no migration: `ChatErrorKind` is persisted only inside the JSONB `chats.last_error` column, whose decoder accepts unknown kinds. The hook docs also correct the tool-input convergence window. A batch dispatches sequentially before the assistant row commits, so the original input stays visible for a span that scales with the number of tool calls in the step rather than a single hook timeout. > This PR was written by Mux, an AI coding agent, on Mike's behalf. |
||
|
Michael Suchacz
|
3227cac217 |
feat: add manual chat compaction via /compact (#27081)
Adds a user-triggered `/compact` action for Coder Agents chats: typing
`/compact` in the composer (or picking it from the `/` trigger menu)
summarizes the conversation so far to free up context window space.
## How it works
- New `POST /api/experimental/chats/{chat}/compact` endpoint
(owner-only, RBAC `ActionUpdate`, excluded from the public API reference
via `x-apidocgen skip`). It marks the chat with a durable one-shot
`chats.compaction_requested_at` signal and moves it `waiting -> running`
via a new `RequestCompaction` state transition; no message row is
inserted. AI Gateway attribution needs no per-request key: generation
preparation resolves the owner's synthetic API key (#27170) like any
other turn.
- `RequestCompaction` hands off chat ownership (clears
`worker_id`/`runner_id`) so a worker acquisition hint is published;
since the transition changes no history, the previous runner could
otherwise miss the request under reordered pubsub delivery.
- The background chat worker picks the chat up like any other turn. A
pending manual request takes precedence over turn completion in the
generation decision, and forces compaction even below the automatic
threshold (and when compaction is disabled via threshold=100). The
commit step consumes the request marker in the same transaction; any
transition that ends the turn clears stale markers.
- The summary triplet reuses the automatic-compaction path, now tagged
with a `source` (`automatic` | `manual`) that is plumbed through
streamed progress parts, persisted tool JSON, and the UI label
("Summarized (manual)").
- Validation order: busy chats reject with 409 (state-machine conflict),
empty/already-compacted chats with 409 "nothing to compact", archived
chats with 400; the owner usage-limit check runs last so no-op requests
surface the specific conflict instead of a limit error.
- Web UI: the `/` trigger menu now has a built-in "Commands" group
listing `/compact`; submit intercepts exactly `/compact` and calls the
endpoint instead of sending a message. A personal or workspace skill
named `compact` takes precedence over the built-in command; while skill
collisions are still resolving, an exact `/compact` submission is
blocked with a retryable hint instead of leaking as message text.
History and queued-message edits are never intercepted. After
compaction, the context usage indicator resets to its unknown state
until the next assistant response reports fresh usage, instead of
showing the stale pre-compaction number.
- codersdk: `ExperimentalClient.CompactChat`.
Worker-path execution (rather than compacting synchronously in the
handler) reuses the existing lock fencing, live "Summarizing..."
streaming, retry accounting, restart resilience, and debug-run
observability. Rationale documented in `coderd/x/chatd/ARCHITECTURE.md`.
## Testing
- State machine: transition-matrix coverage for `RequestCompaction`,
marker lifecycle tests (carried by lease renewals/queue appends, cleared
by terminal transitions, consumed by commit), ownership handoff +
acquisition hint assertions.
- Worker: decision-ordering and forced-compaction unit tests;
active-server end-to-end test (manual compact below threshold produces a
`source=manual` summary, returns to `waiting`, no assistant follow-up;
busy chat rejected).
- API: success, archived, non-owner, RBAC-denied, empty-chat, no-daemon
cases; usage-limit ordering (at-limit owners still get
state/nothing-to-compact conflicts for no-op requests, with marker
rollback).
- Frontend: Storybook play tests for the Commands menu group, submit
intercept, skill-name collision, queued-edit passthrough, and
manual/automatic tool rendering; unit tests for command availability
resolution and the post-compaction context usage reset.
> This PR was created by Mux, an AI coding agent, working on Mike's
behalf.
|
||
|
Michael Suchacz
|
4ed6fcced7 |
refactor(coderd): stop storing chat gateway key IDs and drop the columns (#27171)
> Mux is working on behalf of Mike. ## Summary Stop reading and writing the legacy `api_key_id` columns on chat messages and queued messages, and drop the columns in the same PR. Runtime AI Gateway attribution continues to use the per-user synthetic key introduced by #27170. With the columns gone, `sqlc` generates `database.ChatMessage` and `database.ChatQueuedMessage` without `api_key_id`, so no transitional query scaffolding is needed. Migration `000548` drops the `api_key_id` columns. #27170 already removed their foreign keys, so the down migration re-adds nullable text columns without constraints. Previous column values cannot be restored. Also moves the model config validation in `CreateChat` above the message-building work so a disabled or invalid model fails fast. On main this mattered more: the old ordering minted a synthetic API key before rejecting the request. Deploy note: replicas still running the previous release write `api_key_id` on insert, so chat message inserts on old replicas fail during the rolling window after the column drop. This was previously split across two PRs to avoid that window; per review feedback the split added more churn than it was worth for an experimental surface. Depends on #27170 (merged). |
||
|
Cian Johnston
|
b21e0717d5 |
feat: remove chat chain mode (#26980)
Removes OpenAI Responses "chain mode" from chatd. Closes CODAGT-445.
- Deletes `chatopenai/responses.go` (chain detection, activation, prompt filtering, response ID extraction) and its tests.
- Deletes the `ChainBroken` classification in `chaterror` and the chatloop retry bookkeeping that disabled chain mode mid-generation.
- Drops the `chain_broken` label from the `coderd_chatd_stream_retries_total` metric.
- Stops reading and writing `chat_messages.provider_response_id`
- Deletes the dead `ClearChatMessageProviderResponseIDsByChatID` query. Dropping the column is a follow-up migration.
- Deletes three chatloop hooks no caller sets (`ReloadMessages`, `DisableChainMode`, `PrepareMessages`), the dead `const AgentChatContextSentinelPath`, and stale chain-mode comments.
🤖 Generated by Coder Agents on behalf of @johnstcn.
|
||
|
Hugo Dutka
|
3cde346cbb |
fix(coderd/x/chatd): fix compaction still over limit check (#26377)
Addresses [CODAGT-620](https://linear.app/codercom/issue/CODAGT-620/session-can-get-stuck-at-compaction-with-request-failed). We have logic that checks whether message compaction still leaves the chat over the context limit. We want to abort if it does - if we didn't, we'd get into an endless compaction loop. The check's logic was faulty. This PR changes fixes it. The new flow is: 1. In iteration 1, a chat runner commits a message compaction summary. 2. In iteration 2, the runner submits the newly compacted conversation to the LLM provider in order to generate the next message. 3. In iteration 3, 4, 5, etc., if the conversation needs compaction, the runner looks up the configured context limit and the first assistant message after the last compaction summary. It compares the context usage on that message with the context limit. If the usage is over the limit, it returns an error. |
||
|
Hugo Dutka
|
4debd23cbb |
fix: chatd refactor (#26270)
Implements the chatd stabilization RFC. Combines: - https://github.com/coder/coder/pull/25908 - https://github.com/coder/coder/pull/25923 - https://github.com/coder/coder/pull/26109 - https://github.com/coder/coder/pull/26110 - https://github.com/coder/coder/pull/26111 - https://github.com/coder/coder/pull/26112 |