mirror of
https://github.com/coder/coder.git
synced 2026-09-24 15:04:27 +08:00
feat(site/src): add advisor admin settings UI (#24624)
## Summary Add the admin-only Advisor settings form on the Agents Behavior page, wired to the PR 3 HTTP API via TanStack Query. Admins can enable/disable the advisor, tune per-run caps and max output tokens, pick a reasoning effort, and optionally select a dedicated advisor model. ## Motivation PR 3 introduced the configuration storage and API but no surface to change it. This PR puts the knobs in the existing admin Behavior UI so operators can turn the feature on/off and tune it without touching the database. ## Changes - `site/src/pages/AgentsPage/components/AdvisorSettings.tsx`: the admin-only settings panel. - TanStack Query hooks (`chatAdvisorConfig`, `updateChatAdvisorConfig`) with optimistic cache invalidation on save. - Inputs for `Enabled`, `MaxUsesPerRun`, `MaxOutputTokens`, `ReasoningEffort`, `ModelConfigID`. - Model-override dropdown renders "Use chat model" when unset. - `ZERO_UUID` / `isUnsetModelConfigId` helpers normalize the model-override payload in both directions: the backend serializes the unset sentinel as `"00000000-0000-0000-0000-000000000000"`, and the frontend must both recognize that sentinel on read and never send `""` on write (which would trigger a backend `invalid UUID length: 0`). - Behavior page wiring so the panel appears in the existing settings flow alongside the rest of the agent behavior toggles. ## Stack context This is **PR 6 of 6** in the advisor feature stack. It is stacked directly on **PR 3** (the config API) and is a sibling of PRs 4/5, so it can be reviewed independently of the chat-rendering work. ## Scope / non-goals - Does not change advisor runtime behavior; that is PR 4's responsibility. - No new audit entries for this settings change; it rides on the same storage mechanism as the existing site-configs pattern. ## Validation - `pnpm --filter site lint:types` - Manual dogfood against `./scripts/develop.sh`: toggle Enabled on/off, save, confirm `GET /api/experimental/chats/config/advisor` reflects changes, confirm non-admin users cannot reach the form. --- <details> <summary>📋 Implementation Plan (shared across the advisor stack)</summary> # Plan: Add a Mux-style advisor tool to coder agents/chatd ## Outcome Add a first-class `advisor` tool to agent chats in `coderd/x/chatd` that feels native to Coder: - it is a built-in server-side tool, not an MCP/dynamic-tool workaround; - it performs a nested **tool-less** model call for strategic advice; - it is exposed only when eligible, and the prompt mentions it only when it is actually available; - it is treated as a **planning-only** tool so it does not run alongside action tools in the same batch; - it tracks usage/cost separately enough for operators to reason about it; - it has a minimally polished UI in the Agents page; - and it ships with explicit dogfooding evidence, including screenshots and repro videos. ## Design decisions to lock before coding 1. **Primary architecture:** native built-in tool in `chattool/`, backed by a small `chatadvisor` package. 2. **Nested model execution:** reuse chatd's existing model/provider stack for a one-step, tool-less advisor call rather than inventing a new provider pathway. 3. **Execution policy:** treat `advisor` as an exclusive/planning-only tool; mixed batches must return structured policy errors and force the model to retry cleanly. 4. **Availability:** initial rollout is for root agent chats only; disable for child/sub-agent chats until recursion/cost policy is proven. 5. **Prompt sync:** use one eligibility boolean to drive both tool registration and advisor guidance injection. 6. **Persistence/cost split:** MVP should keep advisor usage visible in result metadata and server metrics; only add DB schema if product/billing explicitly needs queryable advisor-specific cost. 7. **UI scope:** generic tool rendering is an acceptable temporary milestone during backend bring-up, but the release candidate should include a dedicated lightweight advisor renderer. ## Delivery model The work should be executed as coordinated workstreams with one integration owner and parallel contributors for low-conflict areas. The integration owner should own `coderd/x/chatd/chatd.go` because prompt assembly, tool registration, and model resolution all converge there. ## Detailed workstreams ### Repo evidence used for this plan <details> <summary>Mux reference and current chatd seams</summary> **Mux reference implementation** - `src/node/services/tools/advisor.ts` — native advisor tool implementation. - `src/common/constants/advisor.ts` — advisor prompt/constants and truncation policy. - `src/common/utils/tools/tools.ts` — conditional tool registration. - `src/node/services/streamContextBuilder.ts` — injects advisor guidance only when the tool is available. **Current chatd seams** - `coderd/x/chatd/chatd.go` - `processChat()` — tool assembly, prompt assembly, and chatloop invocation. - `resolveChatModel()` — current model/provider/key resolution seam. - `type Config struct` — server-level chatd configuration surface. - `coderd/x/chatd/chatloop/chatloop.go` - `Run()` — main streaming/model loop. - `executeTools()` — built-in tool execution/batching seam. - `coderd/x/chatd/chattool/` — built-in tool implementations. - `site/src/pages/AgentsPage/components/ChatElements/tools/Tool.tsx` — tool renderer dispatch. - `site/src/pages/AgentsPage/components/ChatConversation/messageParsing.ts` and `ConversationTimeline.tsx` — tool/result merge and rendering flow. </details> ### Workstream map and ownership | Workstream | Primary owner | Main files | Can run in parallel? | Done when | |---|---|---|---|---| | 0. Integration + gating | Integration lead | `coderd/x/chatd/chatd.go` | No; central merge lane | Tool registration, prompt sync, and model selection are wired together | | 1. Advisor runtime + tool | Backend agent | new `coderd/x/chatd/chatadvisor/`, new `coderd/x/chatd/chattool/advisor.go` | Yes | Tool can perform a tool-less advisor call in memory and return structured results | | 2. Planning-only execution policy | Chatloop agent | `coderd/x/chatd/chatloop/chatloop.go`, related tests | Yes | Mixed `advisor` + action-tool batches are rejected cleanly and deterministically | | 3. Metrics/usage/config | Backend/telemetry agent | `chatd.go`, `chatloop/metrics.go`, optional config plumbing | Partially; coordinate with integration lead | Advisor usage is separately visible in metadata/metrics and limits are enforced | | 4. Frontend rendering | Frontend agent | `site/.../tools/Tool.tsx`, new `AdvisorTool.tsx`, stories | Yes after result schema stabilizes | Advisor renders as a readable card and story tests pass | | 5. Dogfood + QA evidence | QA agent | dev server, Storybook, dogfood output | After backend + UI are usable | Repro videos, screenshots, and a concise QA report exist | ### Parallelization rules - **Do not split `coderd/x/chatd/chatd.go` across multiple execution agents without an integration lead.** That file owns prompt building, tool registration, model resolution, and cost persistence. - Workstreams 1 and 2 can be developed in parallel and then stacked onto the integration branch. - Workstream 4 should begin once the backend result schema is agreed on, even if the backend is still behind a feature flag. - Any agent that needs to re-check Mux behavior should clone `coder/mux` into a temporary directory (for example, `$(mktemp -d)/mux`) and inspect it read-only; do not vendor or copy code from Mux directly. ## Phase 0 — Preflight and guardrails ### Goals - Align the team on the smallest shippable architecture. - Prevent scope creep into MCP/dynamic-tool/sub-agent variants. - Decide upfront what is MVP vs. follow-up. ### Tasks 1. **Confirm the MVP boundary.** - Ship a built-in advisor tool first. - Do **not** make MCP, dynamic tools, or sub-agents the primary implementation. - Do **not** add transient streaming phases in the first backend PR unless they fall out almost for free. 2. **Confirm local workflow hygiene before coding.** - Ensure the repo is using the project git hooks from `scripts/githooks`. - Do not bypass hooks with `--no-verify`. - Use `./scripts/develop.sh` for the full dev server rather than manual build/run commands. 3. **Lock the model-selection policy.** - **Recommended MVP:** advisor uses the same resolved provider/model/cost config as the current chat, with advisor-specific max-output and usage caps. - **Follow-up only if required:** add a separate `AdvisorModelConfigID`-style override that resolves through the existing `configCache`/model-config path. Do not invent a new free-form `provider:model` parser if chatd already stores provider/model separately. 4. **Lock the persistence policy.** - **Recommended MVP:** no DB migration. Persist advisor-visible metadata in the tool result and record separate metrics in memory/Prometheus. - **Only if product/billing explicitly asks for queryable advisor cost:** add a later DB migration or usage table, following the normal `queries/*.sql` + `make gen` workflow. 5. **Create an execution ADR note in the work item or tracking doc.** - Capture: built-in tool, tool-less nested call, root-chat-only rollout, exclusive execution policy, MVP no-DB-migration default. ### Quality gate - Everyone on the team can state the same answers to these questions: - Is advisor a built-in tool? **Yes.** - Can advisor run with action tools in the same batch? **No.** - Does advisor get tools of its own? **No.** - Is a DB migration required for MVP? **No, unless billing insists.** ## Phase 1 — Build the advisor runtime and tool wrapper ### Goals Create the core advisor implementation in a way that is easy to test and keeps `chattool/` thin. ### Files to add - `coderd/x/chatd/chatadvisor/types.go` - `coderd/x/chatd/chatadvisor/guidance.go` - `coderd/x/chatd/chatadvisor/handoff.go` - `coderd/x/chatd/chatadvisor/runtime.go` - `coderd/x/chatd/chatadvisor/runner.go` - `coderd/x/chatd/chattool/advisor.go` ### Responsibilities by file 1. **`types.go`** - Define the input/result schema used by the tool and UI. - Keep the result shape close to Mux so the UI and model both have predictable cases. - Recommended result variants: - `advice` - `limit_reached` - `error` Recommended shape: ```go type AdvisorArgs struct { Question string `json:"question"` } type AdvisorResult struct { Type string `json:"type"` Advice string `json:"advice,omitempty"` Error string `json:"error,omitempty"` AdvisorModel string `json:"advisor_model,omitempty"` RemainingUses int `json:"remaining_uses,omitempty"` Usage *AdvisorUsageResult `json:"usage,omitempty"` } ``` 2. **`guidance.go`** - Hold two strings: - the nested advisor system prompt; - the parent-agent guidance block to inject into the outer system prompt. - The nested advisor prompt must say, in plain language: - you are advising the parent agent; - you do not address the end user directly; - you do not claim actions happened; - you return concise strategic guidance and tradeoffs. 3. **`runtime.go`** - Define the per-run runtime state. - Recommended fields: - resolved model + model config; - provider keys/options reused from the outer chat; - `MaxUsesPerRun`; - `MaxOutputTokens`; - atomic/current call counter; - callback(s) to obtain the current prompt snapshot and current-step snapshot; - optional metrics/usage hook. - Add fail-fast validation for impossible config: nil model, non-positive limits, empty prompt builders, etc. 4. **`handoff.go`** - Build the advisor handoff message from: - the explicit question; - the exact prompt/messages the parent model just used; - the current step's text/reasoning snapshot, if available; - the most recent relevant tool outputs, if they are already in the prompt snapshot. - **Important:** use the already-prepared outer prompt tail, not a fresh DB reload. That keeps the advisor aligned with compaction and the exact context the outer model saw. - Apply hard truncation budgets with recent-context bias. 5. **`runner.go`** - Execute the nested advisor call. - **Recommended implementation:** call `chatloop.Run()` in an in-memory, one-step mode: - `Tools: nil` - `ProviderTools: nil` - `MaxSteps: 1` - `PersistStep`: capture the assistant output in memory instead of writing DB rows - Reuse the existing provider/model/cost path instead of building a second provider runner. - Assert that no tool definitions are passed to the nested call. 6. **`chattool/advisor.go`** - Keep this file thin and consistent with other built-ins. - Responsibilities: - decode `AdvisorArgs`; - validate `Question` is non-empty and bounded; - call the `chatadvisor` runner; - return a structured tool response. ### Defensive programming requirements - Assert `Question` is non-empty after trimming. - Assert runtime limits are positive. - Assert the nested advisor call runs with zero tools/provider tools. - Assert `AdvisorResult.Type` is one of the known variants before returning. - Assert remaining uses never goes negative. ### Acceptance criteria - A unit test can call the advisor tool with a fake model and receive a stable `advice` result. - The nested advisor call is impossible to run with tools accidentally attached. - The core logic lives in `chatadvisor/`, not embedded inside `chatd.go`. ## Phase 2 — Wire advisor into chatd and keep prompt/tool availability in sync ### Goals Register the tool in the right place, expose it only when eligible, and inject system guidance only when the tool is present. ### Files to modify - `coderd/x/chatd/chatd.go` - optionally a small helper file if `chatd.go` becomes too crowded ### Tasks 1. **Compute one eligibility boolean in `processChat()`.** Recommended inputs: - server-level advisor enabled flag; - root chat only (`chat.ParentChatID == uuid.Nil` or equivalent existing root/child check); - a usable resolved model/provider exists; - optional experiment/workspace/org gate if product wants staged rollout. 2. **Create the runtime once per outer chat run.** - Use the model/config/keys resolved by `resolveChatModel()`. - Reuse provider options from the current chat's `ChatModelCallConfig`. - Set `MaxUsesPerRun` and `MaxOutputTokens` from advisor config defaults. 3. **Register the tool in the built-in tool block.** - Insert after the skill tools and before MCP tools in `processChat()`. - Record `builtinToolNames["advisor"] = true` so metrics stay bounded. 4. **Inject advisor guidance into the outer system prompt using the same boolean.** - Use `chatprompt.InsertSystem()` in the same prompt assembly path that already injects user/system instructions. - Place the block near the existing instruction insertion, before plan-path/skill context blocks. - Wrap the guidance in an explicit tag like `<advisor-guidance>` so it is easy to spot in tests and future refactors. 5. **Keep advisor out of child chats for the first release.** - That avoids recursion/cost blowups with `spawn_agent` / `wait_agent` flows. - Document this explicitly in the rollout notes and tests. ### Acceptance criteria - If advisor is disabled, neither the tool nor the prompt guidance appears. - If advisor is enabled, both the tool and the prompt guidance appear. - Root chats can use advisor; child chats cannot. - Built-in tool names include `advisor` so metrics do not collapse it into the generic `mcp` label. ## Phase 3 — Enforce planning-only execution policy in `chatloop` ### Goals Prevent the model from calling `advisor` and action tools in the same execution batch. ### Files to modify - `coderd/x/chatd/chatloop/chatloop.go` - related chatloop tests ### Recommended implementation Keep the MVP small; do **not** build a general policy engine yet. 1. Add a minimal field to `chatloop.RunOptions`, for example: ```go ExclusiveToolName *string ``` 2. In `Run()` / `executeTools()`, detect the case where the exclusive tool appears in the same local-tool batch as any other locally executed tool. 3. When that happens, synthesize structured tool-result errors for the affected calls instead of executing anything in the batch. - `advisor` should receive a clear error like: _advisor must be called by itself before action tools_. - The sibling action tools should receive a paired policy error like: _this tool was skipped because advisor must run alone_. 4. Let the outer model see those tool errors and retry cleanly. - This is simpler and safer than partial execution or hidden deferral. - It preserves deterministic transcript history for debugging. 5. Pass the just-finished step snapshot into the tool execution context. - The advisor runtime should be able to see the current step's text/reasoning content, because that is often the best hint about what the outer model is trying to decide. ### Why this is the right fit - It matches the intended semantics: advisor is consulted **before** taking action. - It avoids subtle race conditions caused by concurrent built-in tool execution. - It keeps the behavior easy to test with fake models. ### Acceptance criteria - A model-emitted batch containing only `advisor` succeeds. - A model-emitted batch containing `advisor` plus any other locally executed tool returns deterministic policy errors and executes nothing. - Non-advisor tool execution stays unchanged for normal chats. ## Phase 4 — Usage limits, metrics, and configuration ### Goals Make advisor safe to operate without over-designing billing/storage in the first release. ### Files to modify - `coderd/x/chatd/chatd.go` - `coderd/x/chatd/chatloop/metrics.go` as needed - `coderd/x/chatd/chatd.go` `Config` struct and constructor path - optional follow-up config/db files only if a separate advisor model or persistent billing is required ### Tasks 1. **Add explicit server config knobs for MVP.** Recommended fields on `chatd.Config` or a nested advisor config struct: - `AdvisorEnabled bool` - `AdvisorMaxUsesPerRun int` - `AdvisorMaxOutputTokens int64` 2. **Track usage per outer run.** - Reset the counter for each `processChat()` invocation. - Return `remaining_uses` in the tool result. - Return `limit_reached` when the cap is exhausted. 3. **Expose advisor usage metadata in the tool result.** - Include model name and token/cost summary if available. - Use the same `callConfig.Cost` calculation path as the outer chat for MVP if advisor reuses the same model. 4. **Record server-side metrics.** - Count advisor invocations, failures, and latency. - Ensure they show up under the built-in tool label `advisor`. 5. **Optional decision gate: separate advisor model.** - If product insists on a stronger/different advisor model, add a follow-up config hook that resolves another existing chat model config through the same `configCache` path. - Keep that out of the first landing PR unless it is required for acceptance. 6. **Optional decision gate: queryable advisor cost.** - If this becomes required, spin a follow-up DB task: - update `coderd/database/queries/*.sql`; - add migration files; - run `make gen`; - update audit mappings if a new auditable type/field is introduced. ### Acceptance criteria - Advisor calls are capped per outer run. - Limit exhaustion is user-visible in the tool result. - Metrics distinguish advisor calls from other built-in tools. - MVP does not require a schema migration unless explicitly approved. ## Phase 5 — Frontend rendering and Storybook coverage ### Goals Make advisor feel intentional in the Agents UI without blocking the backend on fancy streaming UI. ### Files to modify - `site/src/pages/AgentsPage/components/ChatElements/tools/Tool.tsx` - new `site/src/pages/AgentsPage/components/ChatElements/tools/AdvisorTool.tsx` - Storybook story file(s) in the same tools directory ### Delivery strategy 1. **Intermediate milestone during backend bring-up:** rely on the existing generic tool renderer if needed. - This is acceptable only as a short-lived integration checkpoint. 2. **Release milestone:** add a dedicated lightweight `AdvisorTool` renderer. - Reuse existing primitives: - `ToolCollapsible` - `ToolIcon` - `Response` for markdown/prose rendering - `ScrollArea` if the advice can be long - Keep styling light and consistent with the Agents page. - Do not add unnecessary React memoization in `site/src/pages/AgentsPage/`; that area is already React-Compiler aware. 3. **Render the structured result states cleanly.** - `advice` — readable prose/markdown with optional metadata footer. - `limit_reached` — warning-style message. - `error` — error state with visible fallback text. - `running` — existing tool loading state/spinner is enough for MVP. 4. **Add Storybook coverage instead of ad-hoc component tests.** Recommended stories: - successful advice; - running/loading; - limit reached; - error. 5. **Keep the UI contract narrow.** - Prefer one text field like `advice` plus small metadata rather than a deeply nested schema. - That keeps the UI resilient to prompt iteration. ### Acceptance criteria - The advisor tool card renders readable content rather than raw quoted JSON in the final release branch. - Running, limit, and error states are visibly distinct. - Storybook stories and play assertions cover the new states. - Existing tool rendering flows remain unchanged. ## Phase 6 — Automated tests and validation gates ### Backend tests to add 1. **Advisor runtime/tool tests** - question validation; - tool-less nested execution assertion; - success result shaping; - limit-reached result shaping; - error result shaping. 2. **Prompt/gating tests in chatd** - advisor disabled ⇒ no tool, no guidance; - advisor enabled/root chat ⇒ tool + guidance; - child chat ⇒ advisor absent. 3. **Chatloop policy tests** - advisor alone runs; - advisor + action tool mixed batch returns deterministic policy errors; - non-advisor tools still execute normally. 4. **Usage/metrics tests** - per-run cap resets correctly; - builtin tool labeling includes `advisor`; - returned metadata includes model/usage summary when available. ### Frontend tests to add - Storybook `play()` assertions for the advisor renderer states. - Verify expand/collapse behavior and visible fallback text. - Verify the message timeline still renders adjacent tools correctly. ### Recommended command sequence Run these as the implementation matures, not only at the end: 1. Backend-focused gate after phases 1–4: - `make test RUN=TestAdvisor` - `make test RUN=TestChatloopAdvisor` - `make lint` 2. Frontend-focused gate after phase 5: - `pnpm test:storybook src/pages/AgentsPage/components/ChatElements/tools/AdvisorTool.stories.tsx` - `pnpm lint` - `pnpm format` 3. Final repo gate before handoff: - `make pre-commit` - run any additional targeted `make test RUN=...` selections covering touched chatd paths > Use the exact new test names the implementing agents create; the names above are recommended anchors, not existing tests. ## Dogfooding plan ### Principle Dogfood the change as a real agent feature, not just a unit-tested backend. Per the dogfood and `agent-browser` skills, the reviewer should get **watchable repro videos** plus screenshots that make the behavior obvious without reading logs. ### Required setup 1. Start the full dev environment with: - `./scripts/develop.sh` 2. If the frontend renderer changes, also start Storybook from `site/` with: - `pnpm storybook --no-open` 3. Use `agent-browser` directly — **never `npx agent-browser`**. 4. Use named browser sessions and an output folder such as: - `./dogfood-output/advisor/` - with subfolders `screenshots/` and `videos/` ### Evidence protocol For every interactive scenario below: 1. Start video recording **before** the action. 2. Capture step-by-step screenshots at human pace. 3. Capture one annotated screenshot of the final state. 4. Stop the recording. 5. Note the exact pass/fail observation in the QA report. For static UI states (for example Storybook error/limit cards), an annotated screenshot is sufficient; video is optional but still encouraged by this project’s review preference. ### Dogfood scenarios #### Scenario A — Happy path in the real Agents UI **Goal:** prove that a root agent chat can invoke advisor and produce a readable recommendation before taking further action. Steps: 1. Open the Agents page with an advisor-enabled root chat. 2. Start a repro video. 3. Send a prompt that should reasonably trigger strategic planning, such as an architecture or multi-tradeoff question. 4. Capture screenshots of: - the prompt before send; - the running advisor state; - the completed advisor card and the assistant’s follow-up response. 5. Stop recording. Pass criteria: - advisor appears in the timeline; - the rendered result is readable; - the assistant can continue after consuming the advisor output. #### Scenario B — Advisor unavailable path **Goal:** prove the feature is truly gated. Suggested variants (at least one is required, both are better): - feature flag/config off; - child/sub-agent chat. Evidence: - annotated screenshot of the chat/tool state showing advisor is absent; - short video if toggling the gate live is part of the repro. Pass criteria: - no advisor tool is available; - no advisor-specific prompt behavior leaks through. #### Scenario C — UI states in Storybook **Goal:** prove the renderer handles non-happy states cleanly. Required story states: - success/advice; - running; - limit reached; - error. Evidence: - one screenshot per state; - at least one short video showing collapse/expand behavior. Pass criteria: - success renders readable advice; - limit/error have visible fallback text; - the component behaves like the other tool cards. #### Scenario D — Regression sweep of nearby tools **Goal:** ensure advisor does not break the surrounding chat timeline. Check at minimum: - another existing built-in tool still renders correctly near advisor; - sub-agent/tool cards still expand/collapse normally; - no obvious console errors appear in the Agents page during the advisor flow. Evidence: - screenshots of adjacent tool cards; - console/error capture if anything suspicious appears. ### `agent-browser` usage notes for the QA agent - Prefer `agent-browser batch` for 2+ sequential commands when no intermediate parsing is needed. - Use `snapshot -i` to discover interactive refs. - Re-snapshot after navigation or major DOM changes. - Avoid `wait --load networkidle` unless the page is known to go idle; prefer explicit element/text waits or short fixed waits. - Record videos at human pace and include pauses that a reviewer can follow. ## Rollout plan ### Initial rollout - Gate behind a server-side advisor-enabled flag. - Enable only for selected internal/root agent chats first. - Watch metrics for: - invocation count; - failure rate; - latency; - obvious retry loops. ### Expansion conditions Expand beyond the initial rollout only after the following are true: - mixed-batch policy behavior is stable; - cost impact is understood; - frontend UX is readable in production-like dogfood; - no recursion surprises have appeared with sub-agent flows. ### Explicit non-goals for the first release - advisor inside child/sub-agent chats; - provider-agnostic streaming phase UI; - MCP-based external advisor implementation; - mandatory DB-backed advisor cost reporting. ## Final acceptance checklist - [ ] `advisor` is a built-in chatd tool, not an MCP/dynamic-tool substitute. - [ ] The nested advisor call is tool-less and bounded to one in-memory step. - [ ] One eligibility boolean controls both tool registration and prompt guidance injection. - [ ] Root chats can use advisor; child chats cannot in the initial rollout. - [ ] Mixed advisor/action batches produce deterministic policy errors instead of partial execution. - [ ] Per-run usage caps and limit-reached behavior work. - [ ] Advisor usage is visible in metadata/metrics without forcing a DB migration for MVP. - [ ] The Agents UI has a readable advisor card and Storybook coverage. - [ ] Dogfooding produced screenshots and repro videos for the required scenarios. - [ ] Validation commands (`make lint`, targeted `make test`, Storybook tests, `make pre-commit`) passed before handoff. ## Suggested PR split 1. **PR 1 — Backend foundation** - `chatadvisor/` package - `chattool/advisor.go` - `chatloop` exclusive policy - chatd gating/prompt sync - backend tests 2. **PR 2 — Frontend + QA** - advisor renderer - stories/play assertions - dogfood artifacts and QA notes 3. **PR 3 — Optional follow-ups only if demanded by stakeholders** - separate advisor model override - persistent advisor billing/queryability - transient phase-stream UX </details> --- _Generated with [`mux`](https://github.com/coder/mux) • Model: `anthropic:claude-opus-4-7` • Thinking: `max`_
This commit is contained in:
Thomas Kosiewski
@@ -29,9 +29,11 @@ import {
|
||||
} from "../utils/OneWayWebSocket";
|
||||
import { type FieldError, isApiError } from "./errors";
|
||||
import type {
|
||||
AdvisorConfig,
|
||||
DeleteExternalAuthByIDResponse,
|
||||
DynamicParametersRequest,
|
||||
PostWorkspaceUsageRequest,
|
||||
UpdateAdvisorConfigRequest,
|
||||
UsersRequest,
|
||||
} from "./typesGenerated";
|
||||
import * as TypesGen from "./typesGenerated";
|
||||
@@ -3346,6 +3348,19 @@ class ExperimentalApiMethods {
|
||||
await this.axios.put("/api/experimental/chats/config/desktop-enabled", req);
|
||||
};
|
||||
|
||||
getChatAdvisorConfig = async (): Promise<AdvisorConfig> => {
|
||||
const response = await this.axios.get<AdvisorConfig>(
|
||||
"/api/experimental/chats/config/advisor",
|
||||
);
|
||||
return response.data;
|
||||
};
|
||||
|
||||
updateChatAdvisorConfig = async (
|
||||
req: UpdateAdvisorConfigRequest,
|
||||
): Promise<void> => {
|
||||
await this.axios.put("/api/experimental/chats/config/advisor", req);
|
||||
};
|
||||
|
||||
getChatWorkspaceTTL =
|
||||
async (): Promise<TypesGen.ChatWorkspaceTTLResponse> => {
|
||||
const response = await this.axios.get<TypesGen.ChatWorkspaceTTLResponse>(
|
||||
|
||||
@@ -11,6 +11,8 @@ import {
|
||||
addChildToParentInCache,
|
||||
archiveChat,
|
||||
cancelChatListRefetches,
|
||||
chatAdvisorConfig,
|
||||
chatAdvisorConfigKey,
|
||||
chatCostSummary,
|
||||
chatCostSummaryKey,
|
||||
chatDebugRunsKey,
|
||||
@@ -37,6 +39,7 @@ import {
|
||||
TERMINAL_RUN_STATUSES,
|
||||
unarchiveChat,
|
||||
unpinChat,
|
||||
updateChatAdvisorConfig,
|
||||
updateChatPlanMode,
|
||||
updateChildInParentCache,
|
||||
updateInfiniteChatsCache,
|
||||
@@ -57,6 +60,8 @@ vi.mock("#/api/api", () => ({
|
||||
promoteChatQueuedMessage: vi.fn(),
|
||||
proposeChatTitle: vi.fn(),
|
||||
regenerateChatTitle: vi.fn(),
|
||||
getChatAdvisorConfig: vi.fn(),
|
||||
updateChatAdvisorConfig: vi.fn(),
|
||||
},
|
||||
},
|
||||
}));
|
||||
@@ -124,6 +129,56 @@ const createTestQueryClient = (): QueryClient =>
|
||||
},
|
||||
});
|
||||
|
||||
describe("advisor config query factories", () => {
|
||||
it("builds the advisor config query and delegates to the API", async () => {
|
||||
const advisorConfig: TypesGen.AdvisorConfig = {
|
||||
enabled: true,
|
||||
max_uses_per_run: 5,
|
||||
max_output_tokens: 2048,
|
||||
reasoning_effort: "high",
|
||||
model_config_id: "00000000-0000-0000-0000-000000000000",
|
||||
};
|
||||
vi.mocked(API.experimental.getChatAdvisorConfig).mockResolvedValue(
|
||||
advisorConfig,
|
||||
);
|
||||
|
||||
const query = chatAdvisorConfig();
|
||||
|
||||
expect(query.queryKey).toEqual(chatAdvisorConfigKey);
|
||||
await expect(query.queryFn()).resolves.toEqual(advisorConfig);
|
||||
expect(API.experimental.getChatAdvisorConfig).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("sends the update request and invalidates the advisor config cache", async () => {
|
||||
const queryClient = createTestQueryClient();
|
||||
queryClient.setQueryData(chatAdvisorConfigKey, {
|
||||
enabled: false,
|
||||
max_uses_per_run: 0,
|
||||
max_output_tokens: 0,
|
||||
reasoning_effort: "",
|
||||
model_config_id: "",
|
||||
} as TypesGen.AdvisorConfig);
|
||||
|
||||
const req: TypesGen.UpdateAdvisorConfigRequest = {
|
||||
enabled: true,
|
||||
max_uses_per_run: 5,
|
||||
max_output_tokens: 2048,
|
||||
reasoning_effort: "high",
|
||||
model_config_id: "00000000-0000-0000-0000-000000000000",
|
||||
};
|
||||
vi.mocked(API.experimental.updateChatAdvisorConfig).mockResolvedValue();
|
||||
|
||||
const mutation = updateChatAdvisorConfig(queryClient);
|
||||
await mutation.mutationFn(req);
|
||||
expect(API.experimental.updateChatAdvisorConfig).toHaveBeenCalledWith(req);
|
||||
|
||||
await mutation.onSuccess?.();
|
||||
expect(queryClient.getQueryState(chatAdvisorConfigKey)?.isInvalidated).toBe(
|
||||
true,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe("invalidateChatListQueries", () => {
|
||||
it("invalidates flat and infinite chat list queries", async () => {
|
||||
const queryClient = createTestQueryClient();
|
||||
|
||||
@@ -1330,6 +1330,23 @@ export const updateChatDesktopEnabled = (queryClient: QueryClient) => ({
|
||||
});
|
||||
|
||||
export * from "./chatDebugLogging";
|
||||
export const chatAdvisorConfigKey = ["chat-advisor-config"] as const;
|
||||
|
||||
export const chatAdvisorConfig = () => ({
|
||||
queryKey: chatAdvisorConfigKey,
|
||||
queryFn: (): Promise<TypesGen.AdvisorConfig> =>
|
||||
API.experimental.getChatAdvisorConfig(),
|
||||
});
|
||||
|
||||
export const updateChatAdvisorConfig = (queryClient: QueryClient) => ({
|
||||
mutationFn: (req: TypesGen.UpdateAdvisorConfigRequest) =>
|
||||
API.experimental.updateChatAdvisorConfig(req),
|
||||
onSuccess: async () => {
|
||||
await queryClient.invalidateQueries({
|
||||
queryKey: chatAdvisorConfigKey,
|
||||
});
|
||||
},
|
||||
});
|
||||
|
||||
const chatWorkspaceTTLKey = ["chat-workspace-ttl"] as const;
|
||||
|
||||
|
||||
@@ -1,8 +1,11 @@
|
||||
import type { FC } from "react";
|
||||
import { useMutation, useQuery, useQueryClient } from "react-query";
|
||||
import {
|
||||
chatAdvisorConfig,
|
||||
chatDebugLogging,
|
||||
chatDesktopEnabled,
|
||||
chatModelConfigs,
|
||||
updateChatAdvisorConfig,
|
||||
updateChatDebugLogging,
|
||||
updateChatDesktopEnabled,
|
||||
} from "#/api/queries/chats";
|
||||
@@ -21,12 +24,23 @@ const AgentSettingsExperimentsPage: FC = () => {
|
||||
...chatDebugLogging(),
|
||||
enabled: permissions.editDeploymentConfig,
|
||||
});
|
||||
const advisorConfigQuery = useQuery({
|
||||
...chatAdvisorConfig(),
|
||||
enabled: permissions.editDeploymentConfig,
|
||||
});
|
||||
const modelConfigsQuery = useQuery({
|
||||
...chatModelConfigs(),
|
||||
enabled: permissions.editDeploymentConfig,
|
||||
});
|
||||
const saveDesktopEnabledMutation = useMutation(
|
||||
updateChatDesktopEnabled(queryClient),
|
||||
);
|
||||
const saveDebugLoggingMutation = useMutation(
|
||||
updateChatDebugLogging(queryClient),
|
||||
);
|
||||
const saveAdvisorConfigMutation = useMutation(
|
||||
updateChatAdvisorConfig(queryClient),
|
||||
);
|
||||
|
||||
return (
|
||||
<RequirePermission isFeatureVisible={permissions.editDeploymentConfig}>
|
||||
@@ -39,6 +53,18 @@ const AgentSettingsExperimentsPage: FC = () => {
|
||||
onSaveDebugLogging={saveDebugLoggingMutation.mutate}
|
||||
isSavingDebugLogging={saveDebugLoggingMutation.isPending}
|
||||
isSaveDebugLoggingError={saveDebugLoggingMutation.isError}
|
||||
advisorConfigData={advisorConfigQuery.data}
|
||||
isAdvisorConfigLoading={advisorConfigQuery.isLoading}
|
||||
isAdvisorConfigFetching={advisorConfigQuery.isFetching}
|
||||
isAdvisorConfigLoadError={advisorConfigQuery.isError}
|
||||
modelConfigsData={modelConfigsQuery.data ?? []}
|
||||
modelConfigsError={modelConfigsQuery.error}
|
||||
isLoadingModelConfigs={modelConfigsQuery.isLoading}
|
||||
isFetchingModelConfigs={modelConfigsQuery.isFetching}
|
||||
onSaveAdvisorConfig={saveAdvisorConfigMutation.mutate}
|
||||
isSavingAdvisorConfig={saveAdvisorConfigMutation.isPending}
|
||||
isSaveAdvisorConfigError={saveAdvisorConfigMutation.isError}
|
||||
saveAdvisorConfigError={saveAdvisorConfigMutation.error}
|
||||
/>
|
||||
</RequirePermission>
|
||||
);
|
||||
|
||||
@@ -17,6 +17,24 @@ const baseArgs: AgentSettingsExperimentsPageViewProps = {
|
||||
onSaveDebugLogging: fn(),
|
||||
isSavingDebugLogging: false,
|
||||
isSaveDebugLoggingError: false,
|
||||
advisorConfigData: {
|
||||
enabled: false,
|
||||
max_uses_per_run: 0,
|
||||
max_output_tokens: 0,
|
||||
reasoning_effort: "",
|
||||
model_config_id: "",
|
||||
},
|
||||
isAdvisorConfigLoading: false,
|
||||
isAdvisorConfigFetching: false,
|
||||
isAdvisorConfigLoadError: false,
|
||||
modelConfigsData: [],
|
||||
modelConfigsError: undefined,
|
||||
isLoadingModelConfigs: false,
|
||||
isFetchingModelConfigs: false,
|
||||
onSaveAdvisorConfig: fn(),
|
||||
isSavingAdvisorConfig: false,
|
||||
isSaveAdvisorConfigError: false,
|
||||
saveAdvisorConfigError: undefined,
|
||||
};
|
||||
|
||||
const meta = {
|
||||
|
||||
@@ -2,9 +2,15 @@ import type { FC } from "react";
|
||||
import type { UseMutateFunction } from "react-query";
|
||||
import type * as TypesGen from "#/api/typesGenerated";
|
||||
import { AdminChatDebugLoggingSettings } from "./components/AdminChatDebugLoggingSettings";
|
||||
import { AdvisorSettings } from "./components/AdvisorSettings";
|
||||
import { SectionHeader } from "./components/SectionHeader";
|
||||
import { VirtualDesktopSettings } from "./components/VirtualDesktopSettings";
|
||||
|
||||
interface MutationCallbacks {
|
||||
onSuccess?: () => void;
|
||||
onError?: () => void;
|
||||
}
|
||||
|
||||
export interface AgentSettingsExperimentsPageViewProps {
|
||||
desktopEnabledData: TypesGen.ChatDesktopEnabledResponse | undefined;
|
||||
onSaveDesktopEnabled: UseMutateFunction<
|
||||
@@ -24,6 +30,21 @@ export interface AgentSettingsExperimentsPageViewProps {
|
||||
>;
|
||||
isSavingDebugLogging: boolean;
|
||||
isSaveDebugLoggingError: boolean;
|
||||
advisorConfigData: TypesGen.AdvisorConfig | undefined;
|
||||
isAdvisorConfigLoading: boolean;
|
||||
isAdvisorConfigFetching: boolean;
|
||||
isAdvisorConfigLoadError: boolean;
|
||||
modelConfigsData: readonly TypesGen.ChatModelConfig[];
|
||||
modelConfigsError: unknown;
|
||||
isLoadingModelConfigs: boolean;
|
||||
isFetchingModelConfigs: boolean;
|
||||
onSaveAdvisorConfig: (
|
||||
req: TypesGen.UpdateAdvisorConfigRequest,
|
||||
options?: MutationCallbacks,
|
||||
) => void;
|
||||
isSavingAdvisorConfig: boolean;
|
||||
isSaveAdvisorConfigError: boolean;
|
||||
saveAdvisorConfigError: unknown;
|
||||
}
|
||||
|
||||
export const AgentSettingsExperimentsPageView: FC<
|
||||
@@ -37,6 +58,18 @@ export const AgentSettingsExperimentsPageView: FC<
|
||||
onSaveDebugLogging,
|
||||
isSavingDebugLogging,
|
||||
isSaveDebugLoggingError,
|
||||
advisorConfigData,
|
||||
isAdvisorConfigLoading,
|
||||
isAdvisorConfigFetching,
|
||||
isAdvisorConfigLoadError,
|
||||
modelConfigsData,
|
||||
modelConfigsError,
|
||||
isLoadingModelConfigs,
|
||||
isFetchingModelConfigs,
|
||||
onSaveAdvisorConfig,
|
||||
isSavingAdvisorConfig,
|
||||
isSaveAdvisorConfigError,
|
||||
saveAdvisorConfigError,
|
||||
}) => {
|
||||
return (
|
||||
<div className="flex flex-col gap-8">
|
||||
@@ -50,6 +83,20 @@ export const AgentSettingsExperimentsPageView: FC<
|
||||
isSavingDesktopEnabled={isSavingDesktopEnabled}
|
||||
isSaveDesktopEnabledError={isSaveDesktopEnabledError}
|
||||
/>
|
||||
<AdvisorSettings
|
||||
advisorConfigData={advisorConfigData}
|
||||
isAdvisorConfigLoading={isAdvisorConfigLoading}
|
||||
isAdvisorConfigFetching={isAdvisorConfigFetching}
|
||||
isAdvisorConfigLoadError={isAdvisorConfigLoadError}
|
||||
modelConfigs={modelConfigsData}
|
||||
modelConfigsError={modelConfigsError}
|
||||
isLoadingModelConfigs={isLoadingModelConfigs}
|
||||
isFetchingModelConfigs={isFetchingModelConfigs}
|
||||
onSaveAdvisorConfig={onSaveAdvisorConfig}
|
||||
isSavingAdvisorConfig={isSavingAdvisorConfig}
|
||||
isSaveAdvisorConfigError={isSaveAdvisorConfigError}
|
||||
saveAdvisorConfigError={saveAdvisorConfigError}
|
||||
/>
|
||||
<AdminChatDebugLoggingSettings
|
||||
adminSettings={debugLoggingData}
|
||||
onSaveAdminSetting={onSaveDebugLogging}
|
||||
|
||||
@@ -0,0 +1,828 @@
|
||||
import type { Meta, StoryObj } from "@storybook/react-vite";
|
||||
import { expect, fn, userEvent, waitFor, within } from "storybook/test";
|
||||
import type * as TypesGen from "#/api/typesGenerated";
|
||||
import { AdvisorSettings } from "./AdvisorSettings";
|
||||
|
||||
const nilUUID = "00000000-0000-0000-0000-000000000000";
|
||||
|
||||
const mockModelConfigs: TypesGen.ChatModelConfig[] = [
|
||||
{
|
||||
id: "model-1",
|
||||
provider: "openai",
|
||||
model: "gpt-5",
|
||||
display_name: "GPT-5",
|
||||
enabled: true,
|
||||
is_default: true,
|
||||
context_limit: 200000,
|
||||
compression_threshold: 80,
|
||||
created_at: "2025-01-01T00:00:00Z",
|
||||
updated_at: "2025-01-01T00:00:00Z",
|
||||
},
|
||||
{
|
||||
id: "model-2",
|
||||
provider: "anthropic",
|
||||
model: "claude-sonnet-4",
|
||||
display_name: "Claude Sonnet 4",
|
||||
enabled: true,
|
||||
is_default: false,
|
||||
context_limit: 200000,
|
||||
compression_threshold: 80,
|
||||
created_at: "2025-01-01T00:00:00Z",
|
||||
updated_at: "2025-01-01T00:00:00Z",
|
||||
},
|
||||
{
|
||||
id: "model-3",
|
||||
provider: "openai",
|
||||
model: "gpt-3.5",
|
||||
display_name: "GPT-3.5 (Disabled)",
|
||||
enabled: false,
|
||||
is_default: false,
|
||||
context_limit: 16000,
|
||||
compression_threshold: 60,
|
||||
created_at: "2025-01-01T00:00:00Z",
|
||||
updated_at: "2025-01-01T00:00:00Z",
|
||||
},
|
||||
];
|
||||
|
||||
const defaultAdvisorConfig: TypesGen.AdvisorConfig = {
|
||||
enabled: false,
|
||||
max_uses_per_run: 0,
|
||||
max_output_tokens: 0,
|
||||
reasoning_effort: "",
|
||||
model_config_id: "",
|
||||
};
|
||||
|
||||
const meta = {
|
||||
title: "pages/AgentsPage/AdvisorSettings",
|
||||
component: AdvisorSettings,
|
||||
args: {
|
||||
advisorConfigData: defaultAdvisorConfig,
|
||||
isAdvisorConfigLoading: false,
|
||||
isAdvisorConfigFetching: false,
|
||||
isAdvisorConfigLoadError: false,
|
||||
modelConfigs: mockModelConfigs,
|
||||
modelConfigsError: undefined,
|
||||
isLoadingModelConfigs: false,
|
||||
isFetchingModelConfigs: false,
|
||||
onSaveAdvisorConfig: fn((_req, options) => {
|
||||
options?.onSuccess?.();
|
||||
}),
|
||||
isSavingAdvisorConfig: false,
|
||||
isSaveAdvisorConfigError: false,
|
||||
saveAdvisorConfigError: undefined,
|
||||
},
|
||||
decorators: [
|
||||
(Story) => (
|
||||
<div className="max-w-3xl">
|
||||
<Story />
|
||||
</div>
|
||||
),
|
||||
],
|
||||
} satisfies Meta<typeof AdvisorSettings>;
|
||||
|
||||
export default meta;
|
||||
type Story = StoryObj<typeof meta>;
|
||||
|
||||
export const Default: Story = {
|
||||
play: async ({ canvasElement }) => {
|
||||
const canvas = within(canvasElement);
|
||||
const enableAdvisorSwitch = canvas.getByRole("switch", {
|
||||
name: /Enable advisor/i,
|
||||
});
|
||||
|
||||
expect(
|
||||
canvas.queryByRole("spinbutton", { name: /Max uses per run/i }),
|
||||
).not.toBeInTheDocument();
|
||||
expect(
|
||||
canvas.queryByRole("combobox", { name: /Advisor model/i }),
|
||||
).not.toBeInTheDocument();
|
||||
|
||||
await userEvent.click(enableAdvisorSwitch);
|
||||
|
||||
await waitFor(() => {
|
||||
expect(
|
||||
canvas.getByRole("spinbutton", { name: /Max uses per run/i }),
|
||||
).toBeVisible();
|
||||
expect(
|
||||
canvas.getByRole("combobox", { name: /Advisor model/i }),
|
||||
).toBeVisible();
|
||||
});
|
||||
},
|
||||
};
|
||||
|
||||
export const Enabled: Story = {
|
||||
args: {
|
||||
advisorConfigData: {
|
||||
...defaultAdvisorConfig,
|
||||
enabled: true,
|
||||
},
|
||||
},
|
||||
play: async ({ canvasElement, args }) => {
|
||||
const canvas = within(canvasElement);
|
||||
const body = within(document.body);
|
||||
const maxUsesInput = await canvas.findByRole("spinbutton", {
|
||||
name: /Max uses per run/i,
|
||||
});
|
||||
const maxOutputTokensInput = canvas.getByRole("spinbutton", {
|
||||
name: /Max output tokens/i,
|
||||
});
|
||||
const reasoningEffortSelect = canvas.getByRole("combobox", {
|
||||
name: /Reasoning effort/i,
|
||||
});
|
||||
const advisorModelSelect = canvas.getByRole("combobox", {
|
||||
name: /Advisor model/i,
|
||||
});
|
||||
const saveButton = canvas.getByRole("button", { name: /Save/i });
|
||||
|
||||
expect(saveButton).toBeDisabled();
|
||||
|
||||
await userEvent.clear(maxUsesInput);
|
||||
await userEvent.type(maxUsesInput, "5");
|
||||
await userEvent.clear(maxOutputTokensInput);
|
||||
await userEvent.type(maxOutputTokensInput, "2048");
|
||||
|
||||
await userEvent.click(reasoningEffortSelect);
|
||||
await userEvent.click(await body.findByRole("option", { name: /^High$/i }));
|
||||
|
||||
await userEvent.click(advisorModelSelect);
|
||||
expect(
|
||||
body.queryByRole("option", { name: /GPT-3.5 \(Disabled\)/i }),
|
||||
).not.toBeInTheDocument();
|
||||
await userEvent.click(
|
||||
await body.findByRole("option", { name: /Claude Sonnet 4/i }),
|
||||
);
|
||||
|
||||
await waitFor(() => {
|
||||
expect(saveButton).toBeEnabled();
|
||||
});
|
||||
|
||||
await userEvent.click(saveButton);
|
||||
|
||||
await waitFor(() => {
|
||||
expect(args.onSaveAdvisorConfig).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
const [request, options] = args.onSaveAdvisorConfig.mock.calls[0];
|
||||
expect(request).toEqual({
|
||||
enabled: true,
|
||||
max_uses_per_run: 5,
|
||||
max_output_tokens: 2048,
|
||||
reasoning_effort: "high",
|
||||
model_config_id: "model-2",
|
||||
});
|
||||
expect(typeof options?.onSuccess).toBe("function");
|
||||
|
||||
await waitFor(() => {
|
||||
expect(saveButton).toBeDisabled();
|
||||
});
|
||||
},
|
||||
};
|
||||
|
||||
export const SaveWithUseChatModel: Story = {
|
||||
args: {
|
||||
advisorConfigData: {
|
||||
...defaultAdvisorConfig,
|
||||
enabled: true,
|
||||
},
|
||||
},
|
||||
play: async ({ canvasElement, args }) => {
|
||||
const canvas = within(canvasElement);
|
||||
const maxUsesInput = await canvas.findByRole("spinbutton", {
|
||||
name: /Max uses per run/i,
|
||||
});
|
||||
|
||||
expect(
|
||||
canvas.getByRole("combobox", { name: /Advisor model/i }),
|
||||
).toHaveTextContent(/Use chat model/i);
|
||||
|
||||
await userEvent.clear(maxUsesInput);
|
||||
await userEvent.type(maxUsesInput, "3");
|
||||
|
||||
const saveButton = canvas.getByRole("button", { name: /Save/i });
|
||||
await waitFor(() => {
|
||||
expect(saveButton).toBeEnabled();
|
||||
});
|
||||
await userEvent.click(saveButton);
|
||||
|
||||
await waitFor(() => {
|
||||
expect(args.onSaveAdvisorConfig).toHaveBeenCalled();
|
||||
});
|
||||
const [request] = args.onSaveAdvisorConfig.mock.calls[0];
|
||||
expect(request.model_config_id).toBe(nilUUID);
|
||||
},
|
||||
};
|
||||
|
||||
export const NilUUIDInitialRoundTrip: Story = {
|
||||
args: {
|
||||
advisorConfigData: {
|
||||
...defaultAdvisorConfig,
|
||||
enabled: true,
|
||||
model_config_id: nilUUID,
|
||||
},
|
||||
},
|
||||
play: async ({ canvasElement }) => {
|
||||
const canvas = within(canvasElement);
|
||||
const advisorModelSelect = await canvas.findByRole("combobox", {
|
||||
name: /Advisor model/i,
|
||||
});
|
||||
|
||||
expect(advisorModelSelect).toHaveTextContent(/Use chat model/i);
|
||||
},
|
||||
};
|
||||
|
||||
export const CustomConfig: Story = {
|
||||
args: {
|
||||
advisorConfigData: {
|
||||
enabled: true,
|
||||
max_uses_per_run: 7,
|
||||
max_output_tokens: 8192,
|
||||
reasoning_effort: "medium",
|
||||
model_config_id: "model-2",
|
||||
},
|
||||
},
|
||||
play: async ({ canvasElement }) => {
|
||||
const canvas = within(canvasElement);
|
||||
const maxUsesInput = await canvas.findByRole("spinbutton", {
|
||||
name: /Max uses per run/i,
|
||||
});
|
||||
const maxOutputTokensInput = canvas.getByRole("spinbutton", {
|
||||
name: /Max output tokens/i,
|
||||
});
|
||||
|
||||
expect(maxUsesInput).toHaveValue(7);
|
||||
expect(maxOutputTokensInput).toHaveValue(8192);
|
||||
expect(
|
||||
canvas.getByRole("combobox", { name: /Reasoning effort/i }),
|
||||
).toHaveTextContent(/Medium/i);
|
||||
expect(
|
||||
canvas.getByRole("combobox", { name: /Advisor model/i }),
|
||||
).toHaveTextContent(/Claude Sonnet 4/i);
|
||||
},
|
||||
};
|
||||
|
||||
export const UnavailableSelectedModel: Story = {
|
||||
args: {
|
||||
advisorConfigData: {
|
||||
...defaultAdvisorConfig,
|
||||
enabled: true,
|
||||
model_config_id: "22222222-2222-2222-2222-222222222222",
|
||||
},
|
||||
},
|
||||
play: async ({ canvasElement }) => {
|
||||
const canvas = within(canvasElement);
|
||||
const advisorModelSelect = await canvas.findByRole("combobox", {
|
||||
name: /Advisor model/i,
|
||||
});
|
||||
|
||||
expect(advisorModelSelect).toHaveTextContent(
|
||||
/Unavailable model \(22222222-2222-2222-2222-222222222222\)/i,
|
||||
);
|
||||
},
|
||||
};
|
||||
|
||||
export const Loading: Story = {
|
||||
args: {
|
||||
advisorConfigData: undefined,
|
||||
isAdvisorConfigLoading: true,
|
||||
},
|
||||
play: async ({ canvasElement }) => {
|
||||
const canvas = within(canvasElement);
|
||||
|
||||
expect(
|
||||
canvas.getByRole("switch", { name: /Enable advisor/i }),
|
||||
).toBeDisabled();
|
||||
expect(canvas.getByRole("button", { name: /Save/i })).toBeDisabled();
|
||||
},
|
||||
};
|
||||
|
||||
export const Refetching: Story = {
|
||||
args: {
|
||||
advisorConfigData: {
|
||||
...defaultAdvisorConfig,
|
||||
enabled: true,
|
||||
},
|
||||
isAdvisorConfigFetching: true,
|
||||
},
|
||||
play: async ({ canvasElement }) => {
|
||||
const canvas = within(canvasElement);
|
||||
|
||||
expect(
|
||||
canvas.getByRole("switch", { name: /Enable advisor/i }),
|
||||
).toBeDisabled();
|
||||
expect(
|
||||
canvas.getByRole("spinbutton", { name: /Max uses per run/i }),
|
||||
).toBeDisabled();
|
||||
expect(
|
||||
canvas.getByRole("combobox", { name: /Reasoning effort/i }),
|
||||
).toBeDisabled();
|
||||
expect(canvas.getByRole("button", { name: /Save/i })).toBeDisabled();
|
||||
},
|
||||
};
|
||||
|
||||
export const LoadingModelConfigs: Story = {
|
||||
args: {
|
||||
advisorConfigData: {
|
||||
...defaultAdvisorConfig,
|
||||
enabled: true,
|
||||
model_config_id: "model-2",
|
||||
},
|
||||
modelConfigs: [],
|
||||
isLoadingModelConfigs: true,
|
||||
},
|
||||
play: async ({ canvasElement }) => {
|
||||
const canvas = within(canvasElement);
|
||||
const advisorModelSelect = await canvas.findByRole("combobox", {
|
||||
name: /Advisor model/i,
|
||||
});
|
||||
|
||||
expect(advisorModelSelect).toBeDisabled();
|
||||
expect(advisorModelSelect).toHaveTextContent(/Loading/i);
|
||||
expect(
|
||||
canvas.getByText(/Loading chat model overrides\./i),
|
||||
).toBeInTheDocument();
|
||||
},
|
||||
};
|
||||
|
||||
export const ModelConfigsError: Story = {
|
||||
args: {
|
||||
advisorConfigData: {
|
||||
...defaultAdvisorConfig,
|
||||
enabled: true,
|
||||
model_config_id: "model-2",
|
||||
},
|
||||
modelConfigsError: new Error("fail"),
|
||||
},
|
||||
play: async ({ canvasElement }) => {
|
||||
const canvas = within(canvasElement);
|
||||
const advisorModelSelect = await canvas.findByRole("combobox", {
|
||||
name: /Advisor model/i,
|
||||
});
|
||||
|
||||
expect(advisorModelSelect).toBeDisabled();
|
||||
expect(
|
||||
canvas.getByText(
|
||||
/Model overrides are unavailable\. The current selection will be sent unchanged\./i,
|
||||
),
|
||||
).toBeInTheDocument();
|
||||
},
|
||||
};
|
||||
|
||||
export const ModelConfigsErrorWithUnsetSelection: Story = {
|
||||
args: {
|
||||
advisorConfigData: {
|
||||
...defaultAdvisorConfig,
|
||||
enabled: true,
|
||||
},
|
||||
modelConfigsError: new Error("fail"),
|
||||
},
|
||||
play: async ({ canvasElement }) => {
|
||||
const canvas = within(canvasElement);
|
||||
|
||||
expect(
|
||||
canvas.getByText(
|
||||
/Model overrides are unavailable\. Saving will keep using the chat model\./i,
|
||||
),
|
||||
).toBeInTheDocument();
|
||||
},
|
||||
};
|
||||
|
||||
export const LoadError: Story = {
|
||||
args: {
|
||||
advisorConfigData: undefined,
|
||||
isAdvisorConfigLoadError: true,
|
||||
},
|
||||
play: async ({ canvasElement }) => {
|
||||
const canvas = within(canvasElement);
|
||||
|
||||
expect(
|
||||
canvas.getByText(/Failed to load advisor settings\./i),
|
||||
).toBeInTheDocument();
|
||||
},
|
||||
};
|
||||
|
||||
export const Saving: Story = {
|
||||
args: {
|
||||
advisorConfigData: {
|
||||
...defaultAdvisorConfig,
|
||||
enabled: true,
|
||||
},
|
||||
isSavingAdvisorConfig: true,
|
||||
},
|
||||
play: async ({ canvasElement }) => {
|
||||
const canvas = within(canvasElement);
|
||||
|
||||
expect(
|
||||
canvas.getByRole("switch", { name: /Enable advisor/i }),
|
||||
).toBeDisabled();
|
||||
expect(
|
||||
canvas.getByRole("spinbutton", { name: /Max uses per run/i }),
|
||||
).toBeDisabled();
|
||||
expect(
|
||||
canvas.getByRole("combobox", { name: /Reasoning effort/i }),
|
||||
).toBeDisabled();
|
||||
expect(canvas.getByRole("button", { name: /Save/i })).toBeDisabled();
|
||||
},
|
||||
};
|
||||
|
||||
export const SaveError: Story = {
|
||||
args: {
|
||||
advisorConfigData: {
|
||||
...defaultAdvisorConfig,
|
||||
enabled: true,
|
||||
},
|
||||
isSaveAdvisorConfigError: true,
|
||||
},
|
||||
play: async ({ canvasElement }) => {
|
||||
const canvas = within(canvasElement);
|
||||
|
||||
expect(
|
||||
canvas.getByText(/Failed to save advisor settings\./i),
|
||||
).toBeInTheDocument();
|
||||
},
|
||||
};
|
||||
|
||||
export const SaveErrorWithDetail: Story = {
|
||||
args: {
|
||||
advisorConfigData: {
|
||||
...defaultAdvisorConfig,
|
||||
enabled: true,
|
||||
},
|
||||
isSaveAdvisorConfigError: true,
|
||||
saveAdvisorConfigError: new Error(
|
||||
"reasoning_effort must be one of: low, medium, high.",
|
||||
),
|
||||
},
|
||||
play: async ({ canvasElement }) => {
|
||||
const canvas = within(canvasElement);
|
||||
|
||||
expect(
|
||||
canvas.getByText(/reasoning_effort must be one of: low, medium, high\./i),
|
||||
).toBeInTheDocument();
|
||||
},
|
||||
};
|
||||
|
||||
export const DeselectModelBackToUseChatModel: Story = {
|
||||
args: {
|
||||
advisorConfigData: {
|
||||
...defaultAdvisorConfig,
|
||||
enabled: true,
|
||||
model_config_id: "model-2",
|
||||
},
|
||||
},
|
||||
play: async ({ canvasElement, args }) => {
|
||||
const canvas = within(canvasElement);
|
||||
const body = within(document.body);
|
||||
const advisorModelSelect = await canvas.findByRole("combobox", {
|
||||
name: /Advisor model/i,
|
||||
});
|
||||
|
||||
expect(advisorModelSelect).toHaveTextContent(/Claude Sonnet 4/i);
|
||||
|
||||
await userEvent.click(advisorModelSelect);
|
||||
await userEvent.click(
|
||||
await body.findByRole("option", { name: /^Use chat model$/i }),
|
||||
);
|
||||
|
||||
await waitFor(() => {
|
||||
expect(advisorModelSelect).toHaveTextContent(/Use chat model/i);
|
||||
});
|
||||
|
||||
const saveButton = canvas.getByRole("button", { name: /Save/i });
|
||||
await waitFor(() => {
|
||||
expect(saveButton).toBeEnabled();
|
||||
});
|
||||
await userEvent.click(saveButton);
|
||||
|
||||
await waitFor(() => {
|
||||
expect(args.onSaveAdvisorConfig).toHaveBeenCalled();
|
||||
});
|
||||
const [request] = args.onSaveAdvisorConfig.mock.calls[0];
|
||||
expect(request.model_config_id).toBe(nilUUID);
|
||||
},
|
||||
};
|
||||
|
||||
export const DisableAdvisorWithDeletedModel: Story = {
|
||||
args: {
|
||||
advisorConfigData: {
|
||||
enabled: true,
|
||||
max_uses_per_run: 5,
|
||||
max_output_tokens: 2048,
|
||||
reasoning_effort: "high",
|
||||
model_config_id: "22222222-2222-2222-2222-222222222222",
|
||||
},
|
||||
},
|
||||
play: async ({ canvasElement, args }) => {
|
||||
const canvas = within(canvasElement);
|
||||
const enableAdvisorSwitch = await canvas.findByRole("switch", {
|
||||
name: /Enable advisor/i,
|
||||
});
|
||||
|
||||
expect(
|
||||
canvas.getByRole("combobox", { name: /Advisor model/i }),
|
||||
).toHaveTextContent(
|
||||
/Unavailable model \(22222222-2222-2222-2222-222222222222\)/i,
|
||||
);
|
||||
|
||||
await userEvent.click(enableAdvisorSwitch);
|
||||
|
||||
const saveButton = canvas.getByRole("button", { name: /Save/i });
|
||||
await waitFor(() => {
|
||||
expect(saveButton).toBeEnabled();
|
||||
});
|
||||
await userEvent.click(saveButton);
|
||||
|
||||
await waitFor(() => {
|
||||
expect(args.onSaveAdvisorConfig).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
// The backend rejects unknown non-nil model IDs, so disabling must
|
||||
// scrub the stale override rather than forwarding it and failing
|
||||
// the save with a 400.
|
||||
const [request] = args.onSaveAdvisorConfig.mock.calls[0];
|
||||
expect(request).toEqual({
|
||||
enabled: false,
|
||||
max_uses_per_run: 5,
|
||||
max_output_tokens: 2048,
|
||||
reasoning_effort: "high",
|
||||
model_config_id: nilUUID,
|
||||
});
|
||||
},
|
||||
};
|
||||
|
||||
export const DisableAdvisorWhileModelConfigsLoadingPreservesOverride: Story = {
|
||||
args: {
|
||||
advisorConfigData: {
|
||||
enabled: true,
|
||||
max_uses_per_run: 5,
|
||||
max_output_tokens: 2048,
|
||||
reasoning_effort: "high",
|
||||
model_config_id: "22222222-2222-2222-2222-222222222222",
|
||||
},
|
||||
modelConfigs: [],
|
||||
isLoadingModelConfigs: true,
|
||||
},
|
||||
play: async ({ canvasElement, args }) => {
|
||||
const canvas = within(canvasElement);
|
||||
const enableAdvisorSwitch = await canvas.findByRole("switch", {
|
||||
name: /Enable advisor/i,
|
||||
});
|
||||
|
||||
await userEvent.click(enableAdvisorSwitch);
|
||||
|
||||
const saveButton = canvas.getByRole("button", { name: /Save/i });
|
||||
await waitFor(() => {
|
||||
expect(saveButton).toBeEnabled();
|
||||
});
|
||||
await userEvent.click(saveButton);
|
||||
|
||||
await waitFor(() => {
|
||||
expect(args.onSaveAdvisorConfig).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
// While the model-configs query is in flight we cannot distinguish a
|
||||
// genuinely deleted override from one we simply have not fetched yet,
|
||||
// so the override must be forwarded unchanged. The backend will
|
||||
// surface a specific 400 if the ID really has been deleted.
|
||||
const [request] = args.onSaveAdvisorConfig.mock.calls[0];
|
||||
expect(request).toEqual({
|
||||
enabled: false,
|
||||
max_uses_per_run: 5,
|
||||
max_output_tokens: 2048,
|
||||
reasoning_effort: "high",
|
||||
model_config_id: "22222222-2222-2222-2222-222222222222",
|
||||
});
|
||||
},
|
||||
};
|
||||
|
||||
export const DisableAdvisorWithModelConfigsErrorPreservesOverride: Story = {
|
||||
args: {
|
||||
advisorConfigData: {
|
||||
enabled: true,
|
||||
max_uses_per_run: 5,
|
||||
max_output_tokens: 2048,
|
||||
reasoning_effort: "high",
|
||||
model_config_id: "22222222-2222-2222-2222-222222222222",
|
||||
},
|
||||
modelConfigs: [],
|
||||
modelConfigsError: new Error("fail"),
|
||||
},
|
||||
play: async ({ canvasElement, args }) => {
|
||||
const canvas = within(canvasElement);
|
||||
const enableAdvisorSwitch = await canvas.findByRole("switch", {
|
||||
name: /Enable advisor/i,
|
||||
});
|
||||
|
||||
await userEvent.click(enableAdvisorSwitch);
|
||||
|
||||
const saveButton = canvas.getByRole("button", { name: /Save/i });
|
||||
await waitFor(() => {
|
||||
expect(saveButton).toBeEnabled();
|
||||
});
|
||||
await userEvent.click(saveButton);
|
||||
|
||||
await waitFor(() => {
|
||||
expect(args.onSaveAdvisorConfig).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
// When the model-configs query has failed we cannot verify whether
|
||||
// the override still exists, so the override must be forwarded
|
||||
// unchanged rather than silently dropped. The backend surfaces a
|
||||
// specific 400 if the ID really has been deleted.
|
||||
const [request] = args.onSaveAdvisorConfig.mock.calls[0];
|
||||
expect(request).toEqual({
|
||||
enabled: false,
|
||||
max_uses_per_run: 5,
|
||||
max_output_tokens: 2048,
|
||||
reasoning_effort: "high",
|
||||
model_config_id: "22222222-2222-2222-2222-222222222222",
|
||||
});
|
||||
},
|
||||
};
|
||||
|
||||
export const DisableAdvisorWhileModelConfigsRefetchingPreservesOverride: Story =
|
||||
{
|
||||
args: {
|
||||
advisorConfigData: {
|
||||
enabled: true,
|
||||
max_uses_per_run: 5,
|
||||
max_output_tokens: 2048,
|
||||
reasoning_effort: "high",
|
||||
model_config_id: "22222222-2222-2222-2222-222222222222",
|
||||
},
|
||||
// Simulate a background refetch with stale cached data: react-query
|
||||
// keeps `isLoading` false once cached data exists, so only
|
||||
// `isFetching` flags the in-flight refetch. The cached list does not
|
||||
// contain the override ID.
|
||||
modelConfigs: mockModelConfigs,
|
||||
isLoadingModelConfigs: false,
|
||||
isFetchingModelConfigs: true,
|
||||
},
|
||||
play: async ({ canvasElement, args }) => {
|
||||
const canvas = within(canvasElement);
|
||||
const enableAdvisorSwitch = await canvas.findByRole("switch", {
|
||||
name: /Enable advisor/i,
|
||||
});
|
||||
|
||||
await userEvent.click(enableAdvisorSwitch);
|
||||
|
||||
const saveButton = canvas.getByRole("button", { name: /Save/i });
|
||||
await waitFor(() => {
|
||||
expect(saveButton).toBeEnabled();
|
||||
});
|
||||
await userEvent.click(saveButton);
|
||||
|
||||
await waitFor(() => {
|
||||
expect(args.onSaveAdvisorConfig).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
// While a background refetch is in flight the cached model list may
|
||||
// be stale, so the scrub guard must treat the absence of the
|
||||
// override as indeterminate and forward the ID unchanged. Otherwise
|
||||
// a still-valid override could be silently dropped just because the
|
||||
// cache lags behind the server.
|
||||
const [request] = args.onSaveAdvisorConfig.mock.calls[0];
|
||||
expect(request).toEqual({
|
||||
enabled: false,
|
||||
max_uses_per_run: 5,
|
||||
max_output_tokens: 2048,
|
||||
reasoning_effort: "high",
|
||||
model_config_id: "22222222-2222-2222-2222-222222222222",
|
||||
});
|
||||
},
|
||||
};
|
||||
|
||||
export const DisableAdvisorWithDeletedModelAndEmptyModelConfigs: Story = {
|
||||
args: {
|
||||
advisorConfigData: {
|
||||
enabled: true,
|
||||
max_uses_per_run: 5,
|
||||
max_output_tokens: 2048,
|
||||
reasoning_effort: "high",
|
||||
model_config_id: "22222222-2222-2222-2222-222222222222",
|
||||
},
|
||||
modelConfigs: [],
|
||||
modelConfigsError: undefined,
|
||||
isLoadingModelConfigs: false,
|
||||
},
|
||||
play: async ({ canvasElement, args }) => {
|
||||
const canvas = within(canvasElement);
|
||||
const enableAdvisorSwitch = await canvas.findByRole("switch", {
|
||||
name: /Enable advisor/i,
|
||||
});
|
||||
|
||||
await userEvent.click(enableAdvisorSwitch);
|
||||
|
||||
const saveButton = canvas.getByRole("button", { name: /Save/i });
|
||||
await waitFor(() => {
|
||||
expect(saveButton).toBeEnabled();
|
||||
});
|
||||
await userEvent.click(saveButton);
|
||||
|
||||
await waitFor(() => {
|
||||
expect(args.onSaveAdvisorConfig).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
// An empty model-configs list after a successful load is a definitive
|
||||
// answer that the override no longer exists, so disabling must scrub
|
||||
// the stale ID rather than forwarding it and failing the save with a
|
||||
// 400. This covers the recovery case where every model config has
|
||||
// been deleted.
|
||||
const [request] = args.onSaveAdvisorConfig.mock.calls[0];
|
||||
expect(request).toEqual({
|
||||
enabled: false,
|
||||
max_uses_per_run: 5,
|
||||
max_output_tokens: 2048,
|
||||
reasoning_effort: "high",
|
||||
model_config_id: nilUUID,
|
||||
});
|
||||
},
|
||||
};
|
||||
|
||||
export const ValidationBlocksSave: Story = {
|
||||
args: {
|
||||
advisorConfigData: {
|
||||
...defaultAdvisorConfig,
|
||||
enabled: true,
|
||||
max_uses_per_run: 5,
|
||||
},
|
||||
},
|
||||
play: async ({ canvasElement }) => {
|
||||
const canvas = within(canvasElement);
|
||||
const maxUsesInput = await canvas.findByRole("spinbutton", {
|
||||
name: /Max uses per run/i,
|
||||
});
|
||||
const saveButton = canvas.getByRole("button", { name: /Save/i });
|
||||
|
||||
expect(saveButton).toBeDisabled();
|
||||
|
||||
// Dirty-but-invalid state: the field is blank, so client validation
|
||||
// must keep Save disabled even though the form is now dirty.
|
||||
await userEvent.clear(maxUsesInput);
|
||||
await waitFor(() => {
|
||||
expect(saveButton).toBeDisabled();
|
||||
});
|
||||
|
||||
await userEvent.type(maxUsesInput, "3");
|
||||
await waitFor(() => {
|
||||
expect(saveButton).toBeEnabled();
|
||||
});
|
||||
},
|
||||
};
|
||||
|
||||
export const DisableThenSave: Story = {
|
||||
args: {
|
||||
advisorConfigData: {
|
||||
enabled: true,
|
||||
max_uses_per_run: 5,
|
||||
max_output_tokens: 2048,
|
||||
reasoning_effort: "high",
|
||||
model_config_id: "model-2",
|
||||
},
|
||||
},
|
||||
play: async ({ canvasElement, args }) => {
|
||||
const canvas = within(canvasElement);
|
||||
const enableAdvisorSwitch = await canvas.findByRole("switch", {
|
||||
name: /Enable advisor/i,
|
||||
});
|
||||
|
||||
expect(
|
||||
canvas.getByRole("spinbutton", { name: /Max uses per run/i }),
|
||||
).toBeVisible();
|
||||
|
||||
// Clear a numeric field to an invalid value before disabling. After
|
||||
// disabling the field is hidden, and saving must not silently overwrite
|
||||
// the stored limit with a coerced value.
|
||||
const maxUsesInput = canvas.getByRole("spinbutton", {
|
||||
name: /Max uses per run/i,
|
||||
});
|
||||
await userEvent.clear(maxUsesInput);
|
||||
|
||||
await userEvent.click(enableAdvisorSwitch);
|
||||
|
||||
await waitFor(() => {
|
||||
expect(
|
||||
canvas.queryByRole("spinbutton", { name: /Max uses per run/i }),
|
||||
).not.toBeInTheDocument();
|
||||
});
|
||||
|
||||
const saveButton = canvas.getByRole("button", { name: /Save/i });
|
||||
await waitFor(() => {
|
||||
expect(saveButton).toBeEnabled();
|
||||
});
|
||||
|
||||
await userEvent.click(saveButton);
|
||||
|
||||
await waitFor(() => {
|
||||
expect(args.onSaveAdvisorConfig).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
const [request] = args.onSaveAdvisorConfig.mock.calls[0];
|
||||
expect(request).toEqual({
|
||||
enabled: false,
|
||||
max_uses_per_run: 5,
|
||||
max_output_tokens: 2048,
|
||||
reasoning_effort: "high",
|
||||
model_config_id: "model-2",
|
||||
});
|
||||
},
|
||||
};
|
||||
@@ -0,0 +1,498 @@
|
||||
import { useFormik } from "formik";
|
||||
import { TriangleAlertIcon } from "lucide-react";
|
||||
import { type FC, useEffect, useId, useRef } from "react";
|
||||
import { getErrorMessage } from "#/api/errors";
|
||||
import type {
|
||||
AdvisorConfig,
|
||||
ChatModelConfig,
|
||||
UpdateAdvisorConfigRequest,
|
||||
} from "#/api/typesGenerated";
|
||||
import { Badge } from "#/components/Badge/Badge";
|
||||
import { Button } from "#/components/Button/Button";
|
||||
import { Input } from "#/components/Input/Input";
|
||||
import { Label } from "#/components/Label/Label";
|
||||
import {
|
||||
Select,
|
||||
SelectContent,
|
||||
SelectItem,
|
||||
SelectTrigger,
|
||||
SelectValue,
|
||||
} from "#/components/Select/Select";
|
||||
import { Switch } from "#/components/Switch/Switch";
|
||||
|
||||
const nilUUID = "00000000-0000-0000-0000-000000000000";
|
||||
const advisorReasoningEfforts = ["", "low", "medium", "high"] as const;
|
||||
type AdvisorReasoningEffort = (typeof advisorReasoningEfforts)[number];
|
||||
const chatModelFallbackValue = "__use-chat-model__";
|
||||
const unavailableModelValue = "__unavailable-model__";
|
||||
const chatReasoningFallbackValue = "__use-chat-reasoning__";
|
||||
|
||||
interface MutationCallbacks {
|
||||
onSuccess?: () => void;
|
||||
onError?: () => void;
|
||||
}
|
||||
|
||||
interface AdvisorSettingsProps {
|
||||
advisorConfigData: AdvisorConfig | undefined;
|
||||
isAdvisorConfigLoading: boolean;
|
||||
isAdvisorConfigFetching: boolean;
|
||||
isAdvisorConfigLoadError: boolean;
|
||||
modelConfigs: readonly ChatModelConfig[];
|
||||
modelConfigsError: unknown;
|
||||
isLoadingModelConfigs: boolean;
|
||||
isFetchingModelConfigs: boolean;
|
||||
onSaveAdvisorConfig: (
|
||||
req: UpdateAdvisorConfigRequest,
|
||||
options?: MutationCallbacks,
|
||||
) => void;
|
||||
isSavingAdvisorConfig: boolean;
|
||||
isSaveAdvisorConfigError: boolean;
|
||||
saveAdvisorConfigError: unknown;
|
||||
}
|
||||
|
||||
type AdvisorSettingsFormValues = {
|
||||
enabled: boolean;
|
||||
max_uses_per_run: string;
|
||||
max_output_tokens: string;
|
||||
reasoning_effort: AdvisorReasoningEffort;
|
||||
model_config_id: string;
|
||||
};
|
||||
|
||||
const isUnsetModelConfigId = (id: string): boolean =>
|
||||
id === "" || id === nilUUID;
|
||||
|
||||
const isAdvisorReasoningEffort = (
|
||||
value: string,
|
||||
): value is AdvisorReasoningEffort => {
|
||||
return advisorReasoningEfforts.includes(value as AdvisorReasoningEffort);
|
||||
};
|
||||
|
||||
const normalizeNonNegativeInteger = (
|
||||
value: number | string | undefined,
|
||||
): number => {
|
||||
const parsed = typeof value === "number" ? value : Number(value);
|
||||
if (!Number.isFinite(parsed) || parsed < 0) {
|
||||
return 0;
|
||||
}
|
||||
return Math.trunc(parsed);
|
||||
};
|
||||
|
||||
const normalizeAdvisorConfig = (
|
||||
config: AdvisorConfig | undefined,
|
||||
): AdvisorSettingsFormValues => {
|
||||
const reasoningEffort = config?.reasoning_effort ?? "";
|
||||
return {
|
||||
enabled: config?.enabled ?? false,
|
||||
max_uses_per_run: String(
|
||||
normalizeNonNegativeInteger(config?.max_uses_per_run),
|
||||
),
|
||||
max_output_tokens: String(
|
||||
normalizeNonNegativeInteger(config?.max_output_tokens),
|
||||
),
|
||||
reasoning_effort: isAdvisorReasoningEffort(reasoningEffort)
|
||||
? reasoningEffort
|
||||
: "",
|
||||
model_config_id:
|
||||
typeof config?.model_config_id === "string" &&
|
||||
!isUnsetModelConfigId(config.model_config_id)
|
||||
? config.model_config_id
|
||||
: "",
|
||||
};
|
||||
};
|
||||
|
||||
const toAdvisorConfigRequest = (
|
||||
values: AdvisorSettingsFormValues,
|
||||
): UpdateAdvisorConfigRequest => ({
|
||||
enabled: values.enabled,
|
||||
max_uses_per_run: normalizeNonNegativeInteger(values.max_uses_per_run),
|
||||
max_output_tokens: normalizeNonNegativeInteger(values.max_output_tokens),
|
||||
reasoning_effort: values.reasoning_effort,
|
||||
model_config_id: isUnsetModelConfigId(values.model_config_id)
|
||||
? nilUUID
|
||||
: values.model_config_id,
|
||||
});
|
||||
|
||||
const isNonNegativeIntegerString = (value: string): boolean => {
|
||||
if (value.trim() === "") {
|
||||
return false;
|
||||
}
|
||||
const parsed = Number(value);
|
||||
return Number.isFinite(parsed) && parsed >= 0 && Number.isInteger(parsed);
|
||||
};
|
||||
|
||||
const validateAdvisorConfig = (values: AdvisorSettingsFormValues) => {
|
||||
const errors: Partial<Record<keyof AdvisorSettingsFormValues, string>> = {};
|
||||
|
||||
// Skip validation of the advisor-only fields when the feature is disabled.
|
||||
// Those inputs are hidden, so an admin disabling the advisor should not be
|
||||
// blocked by stale invalid values left in hidden fields.
|
||||
if (!values.enabled) {
|
||||
return errors;
|
||||
}
|
||||
|
||||
if (!isNonNegativeIntegerString(values.max_uses_per_run)) {
|
||||
errors.max_uses_per_run =
|
||||
"Max uses per run must be a non-negative integer.";
|
||||
}
|
||||
|
||||
if (!isNonNegativeIntegerString(values.max_output_tokens)) {
|
||||
errors.max_output_tokens =
|
||||
"Max output tokens must be a non-negative integer.";
|
||||
}
|
||||
|
||||
if (!isAdvisorReasoningEffort(values.reasoning_effort)) {
|
||||
errors.reasoning_effort = "Select a valid reasoning effort.";
|
||||
}
|
||||
|
||||
return errors;
|
||||
};
|
||||
|
||||
const getModelDisplayName = (config: ChatModelConfig): string =>
|
||||
config.display_name.trim() || config.model;
|
||||
|
||||
const getReasoningEffortLabel = (value: AdvisorReasoningEffort): string => {
|
||||
switch (value) {
|
||||
case "low":
|
||||
return "Low";
|
||||
case "medium":
|
||||
return "Medium";
|
||||
case "high":
|
||||
return "High";
|
||||
default:
|
||||
return "Use chat model default";
|
||||
}
|
||||
};
|
||||
|
||||
export const AdvisorSettings: FC<AdvisorSettingsProps> = ({
|
||||
advisorConfigData,
|
||||
isAdvisorConfigLoading,
|
||||
isAdvisorConfigFetching,
|
||||
isAdvisorConfigLoadError,
|
||||
modelConfigs,
|
||||
modelConfigsError,
|
||||
isLoadingModelConfigs,
|
||||
isFetchingModelConfigs,
|
||||
onSaveAdvisorConfig,
|
||||
isSavingAdvisorConfig,
|
||||
isSaveAdvisorConfigError,
|
||||
saveAdvisorConfigError,
|
||||
}) => {
|
||||
const maxUsesId = useId();
|
||||
const maxOutputTokensId = useId();
|
||||
const hasLoadedAdvisorConfig = advisorConfigData !== undefined;
|
||||
const enabledModelConfigs = modelConfigs.filter((config) => config.enabled);
|
||||
|
||||
// Track the most recent committed advisor values (the server's view or the
|
||||
// last successful save). Reading `advisorConfigData` directly in `onSubmit`
|
||||
// can yield a stale snapshot when a refetch is in flight or has failed,
|
||||
// which would silently roll back recently saved limits if the user then
|
||||
// disables the advisor before the query settles.
|
||||
const committedValuesRef = useRef<AdvisorSettingsFormValues>(
|
||||
normalizeAdvisorConfig(advisorConfigData),
|
||||
);
|
||||
useEffect(() => {
|
||||
committedValuesRef.current = normalizeAdvisorConfig(advisorConfigData);
|
||||
}, [advisorConfigData]);
|
||||
|
||||
const form = useFormik<AdvisorSettingsFormValues>({
|
||||
enableReinitialize: true,
|
||||
validateOnMount: true,
|
||||
initialValues: normalizeAdvisorConfig(advisorConfigData),
|
||||
validate: validateAdvisorConfig,
|
||||
onSubmit: (values, { resetForm }) => {
|
||||
// When disabling, preserve the last committed values for the hidden
|
||||
// fields so potentially invalid in-flight edits (empty strings,
|
||||
// fractional numbers) cannot silently overwrite previously
|
||||
// configured limits, and so a pending or failed refetch of the
|
||||
// advisor config cannot revert recently saved values.
|
||||
let source: AdvisorSettingsFormValues = values.enabled
|
||||
? values
|
||||
: { ...committedValuesRef.current, enabled: false };
|
||||
// If the last committed model override references a model config
|
||||
// that no longer exists, the backend rejects the stale ID with a
|
||||
// 400. When disabling, clear the override so a simple disable
|
||||
// stays reliable in that edge case; the override is unusable
|
||||
// anyway and the admin will reselect one on re-enable. Only scrub
|
||||
// when model configs have loaded successfully and no refetch is in
|
||||
// flight: during an initial load, a background refetch, or on
|
||||
// error we cannot distinguish "truly missing" from "not loaded
|
||||
// yet", and deciding from stale cache could either preserve a
|
||||
// now-deleted ID (causing a 400 on disable/save) or silently drop
|
||||
// an override that is actually still valid but missing from a
|
||||
// stale cache. `isLoading` alone is insufficient because
|
||||
// react-query keeps it false during background refetches when
|
||||
// cached data already exists, so `isFetching` covers that gap. An
|
||||
// empty list after a successful load is a definitive answer, so
|
||||
// the scrub still fires (covers the recovery case where every
|
||||
// model config has been deleted).
|
||||
if (
|
||||
!source.enabled &&
|
||||
!isUnsetModelConfigId(source.model_config_id) &&
|
||||
!isLoadingModelConfigs &&
|
||||
!isFetchingModelConfigs &&
|
||||
!modelConfigsError &&
|
||||
!modelConfigs.some((config) => config.id === source.model_config_id)
|
||||
) {
|
||||
source = { ...source, model_config_id: "" };
|
||||
}
|
||||
const request = toAdvisorConfigRequest(source);
|
||||
onSaveAdvisorConfig(request, {
|
||||
onSuccess: () => {
|
||||
const nextValues = normalizeAdvisorConfig(request);
|
||||
committedValuesRef.current = nextValues;
|
||||
resetForm({ values: nextValues });
|
||||
},
|
||||
});
|
||||
},
|
||||
});
|
||||
|
||||
const isFormDisabled =
|
||||
isSavingAdvisorConfig ||
|
||||
isAdvisorConfigLoading ||
|
||||
isAdvisorConfigFetching ||
|
||||
!hasLoadedAdvisorConfig;
|
||||
const isModelSelectDisabled =
|
||||
isFormDisabled || isLoadingModelConfigs || Boolean(modelConfigsError);
|
||||
const hasUnavailableSelectedModel =
|
||||
!isLoadingModelConfigs &&
|
||||
!isUnsetModelConfigId(form.values.model_config_id) &&
|
||||
!enabledModelConfigs.some(
|
||||
(config) => config.id === form.values.model_config_id,
|
||||
);
|
||||
const selectedModelConfig = modelConfigs.find(
|
||||
(config) => config.id === form.values.model_config_id,
|
||||
);
|
||||
const selectedModelLabel = isUnsetModelConfigId(form.values.model_config_id)
|
||||
? "Use chat model"
|
||||
: isLoadingModelConfigs
|
||||
? "Loading..."
|
||||
: selectedModelConfig
|
||||
? getModelDisplayName(selectedModelConfig)
|
||||
: `Unavailable model (${form.values.model_config_id})`;
|
||||
const selectedModelValue = isUnsetModelConfigId(form.values.model_config_id)
|
||||
? chatModelFallbackValue
|
||||
: hasUnavailableSelectedModel
|
||||
? unavailableModelValue
|
||||
: form.values.model_config_id;
|
||||
const modelHelperText = isLoadingModelConfigs
|
||||
? "Loading chat model overrides."
|
||||
: modelConfigsError
|
||||
? isUnsetModelConfigId(form.values.model_config_id)
|
||||
? "Model overrides are unavailable. Saving will keep using the chat model."
|
||||
: "Model overrides are unavailable. The current selection will be sent unchanged."
|
||||
: "Choose a dedicated advisor model, or leave this unset to reuse the chat model.";
|
||||
|
||||
return (
|
||||
<form className="space-y-3" onSubmit={form.handleSubmit}>
|
||||
<div className="flex items-center gap-2">
|
||||
<h3 className="m-0 text-sm font-semibold text-content-primary">
|
||||
Advisor
|
||||
</h3>
|
||||
<Badge size="sm" variant="warning" className="cursor-default">
|
||||
<TriangleAlertIcon className="h-3 w-3" />
|
||||
Experimental feature
|
||||
</Badge>
|
||||
</div>
|
||||
<div className="flex items-center justify-between gap-4">
|
||||
<div className="!mt-0.5 m-0 flex-1 space-y-2 text-xs text-content-secondary">
|
||||
<p className="m-0">
|
||||
Allow root agent chats to call the advisor tool for strategic
|
||||
guidance.
|
||||
</p>
|
||||
<p className="m-0">
|
||||
When enabled, you can cap advisor usage per run and optionally use
|
||||
an override model.
|
||||
</p>
|
||||
</div>
|
||||
<Switch
|
||||
checked={form.values.enabled}
|
||||
onCheckedChange={(checked) =>
|
||||
void form.setFieldValue("enabled", checked)
|
||||
}
|
||||
aria-label="Enable advisor"
|
||||
disabled={isFormDisabled}
|
||||
/>
|
||||
</div>
|
||||
|
||||
{form.values.enabled && (
|
||||
<div className="grid gap-4 rounded-lg border border-border bg-surface-secondary p-4 md:grid-cols-2">
|
||||
<div className="space-y-1.5">
|
||||
<Label htmlFor={maxUsesId} className="text-xs text-content-primary">
|
||||
Max uses per run
|
||||
</Label>
|
||||
<Input
|
||||
id={maxUsesId}
|
||||
name="max_uses_per_run"
|
||||
type="number"
|
||||
min={0}
|
||||
step={1}
|
||||
inputMode="numeric"
|
||||
aria-label="Max uses per run"
|
||||
value={form.values.max_uses_per_run}
|
||||
// Bypass Formik's `handleChange` on purpose: for `type="number"`
|
||||
// it parses the raw input with `parseFloat` and replaces the
|
||||
// declared `string` form value with a `number`, which would
|
||||
// break string-only validators like `isNonNegativeIntegerString`.
|
||||
onChange={(event) =>
|
||||
void form.setFieldValue(
|
||||
"max_uses_per_run",
|
||||
event.currentTarget.value,
|
||||
)
|
||||
}
|
||||
onBlur={form.handleBlur}
|
||||
aria-invalid={Boolean(form.errors.max_uses_per_run)}
|
||||
disabled={isFormDisabled}
|
||||
className="h-9 bg-surface-primary text-[13px]"
|
||||
/>
|
||||
<p className="m-0 text-xs text-content-secondary">
|
||||
Set to 0 to leave the per-run call count unlimited.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div className="space-y-1.5">
|
||||
<Label
|
||||
htmlFor={maxOutputTokensId}
|
||||
className="text-xs text-content-primary"
|
||||
>
|
||||
Max output tokens
|
||||
</Label>
|
||||
<Input
|
||||
id={maxOutputTokensId}
|
||||
name="max_output_tokens"
|
||||
type="number"
|
||||
min={0}
|
||||
step={1}
|
||||
inputMode="numeric"
|
||||
aria-label="Max output tokens"
|
||||
value={form.values.max_output_tokens}
|
||||
// See `max_uses_per_run` above for why `handleChange` is
|
||||
// bypassed: Formik's `type="number"` coercion would replace
|
||||
// the declared `string` form value with a `number`.
|
||||
onChange={(event) =>
|
||||
void form.setFieldValue(
|
||||
"max_output_tokens",
|
||||
event.currentTarget.value,
|
||||
)
|
||||
}
|
||||
onBlur={form.handleBlur}
|
||||
aria-invalid={Boolean(form.errors.max_output_tokens)}
|
||||
disabled={isFormDisabled}
|
||||
className="h-9 bg-surface-primary text-[13px]"
|
||||
/>
|
||||
<p className="m-0 text-xs text-content-secondary">
|
||||
Set to 0 to use the server default output limit.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div className="space-y-1.5">
|
||||
<Label className="text-xs text-content-primary">
|
||||
Reasoning effort
|
||||
</Label>
|
||||
<Select
|
||||
value={form.values.reasoning_effort || chatReasoningFallbackValue}
|
||||
onValueChange={(value) =>
|
||||
void form.setFieldValue(
|
||||
"reasoning_effort",
|
||||
value === chatReasoningFallbackValue ? "" : value,
|
||||
)
|
||||
}
|
||||
disabled={isFormDisabled}
|
||||
>
|
||||
<SelectTrigger
|
||||
className="h-9 bg-surface-primary text-[13px]"
|
||||
aria-label="Reasoning effort"
|
||||
>
|
||||
<SelectValue placeholder="Use chat model default">
|
||||
{getReasoningEffortLabel(form.values.reasoning_effort)}
|
||||
</SelectValue>
|
||||
</SelectTrigger>
|
||||
<SelectContent>
|
||||
<SelectItem value={chatReasoningFallbackValue}>
|
||||
Use chat model default
|
||||
</SelectItem>
|
||||
<SelectItem value="low">Low</SelectItem>
|
||||
<SelectItem value="medium">Medium</SelectItem>
|
||||
<SelectItem value="high">High</SelectItem>
|
||||
</SelectContent>
|
||||
</Select>
|
||||
<p className="m-0 text-xs text-content-secondary">
|
||||
Controls how hard the advisor model reasons before responding.
|
||||
Leave unset to use the model's default.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div className="space-y-1.5">
|
||||
<Label className="text-xs text-content-primary">
|
||||
Advisor model
|
||||
</Label>
|
||||
<Select
|
||||
value={selectedModelValue}
|
||||
onValueChange={(value) => {
|
||||
if (value === chatModelFallbackValue) {
|
||||
void form.setFieldValue("model_config_id", "");
|
||||
return;
|
||||
}
|
||||
if (value === unavailableModelValue) {
|
||||
return;
|
||||
}
|
||||
void form.setFieldValue("model_config_id", value);
|
||||
}}
|
||||
disabled={isModelSelectDisabled}
|
||||
>
|
||||
<SelectTrigger
|
||||
className="h-9 bg-surface-primary text-[13px]"
|
||||
aria-label="Advisor model"
|
||||
>
|
||||
<SelectValue placeholder="Use chat model">
|
||||
{selectedModelLabel}
|
||||
</SelectValue>
|
||||
</SelectTrigger>
|
||||
<SelectContent>
|
||||
{hasUnavailableSelectedModel && (
|
||||
<SelectItem value={unavailableModelValue}>
|
||||
{selectedModelLabel}
|
||||
</SelectItem>
|
||||
)}
|
||||
<SelectItem value={chatModelFallbackValue}>
|
||||
Use chat model
|
||||
</SelectItem>
|
||||
{enabledModelConfigs.map((config) => (
|
||||
<SelectItem key={config.id} value={config.id}>
|
||||
{getModelDisplayName(config)}
|
||||
</SelectItem>
|
||||
))}
|
||||
</SelectContent>
|
||||
</Select>
|
||||
<p className="m-0 text-xs text-content-secondary">
|
||||
{modelHelperText}
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
<div className="flex justify-end">
|
||||
<Button
|
||||
size="sm"
|
||||
type="submit"
|
||||
disabled={isFormDisabled || !form.dirty || !form.isValid}
|
||||
>
|
||||
Save
|
||||
</Button>
|
||||
</div>
|
||||
|
||||
{isSaveAdvisorConfigError && (
|
||||
<p className="m-0 text-xs text-content-destructive">
|
||||
{getErrorMessage(
|
||||
saveAdvisorConfigError,
|
||||
"Failed to save advisor settings.",
|
||||
)}
|
||||
</p>
|
||||
)}
|
||||
{isAdvisorConfigLoadError && (
|
||||
<p className="m-0 text-xs text-content-destructive">
|
||||
Failed to load advisor settings.
|
||||
</p>
|
||||
)}
|
||||
</form>
|
||||
);
|
||||
};
|
||||
Reference in New Issue
Block a user