5 Commits

Include renames

Author	SHA1	Message	Date
Kyle Carberry	2f8bba792a	feat: push MCP server context and tools from agentcontext (#26533) ... ## What Live MCP servers and their tools now flow into the `agentcontext` snapshot and are pushed to coderd via `PushContextState`, stored alongside instruction files and skills. Previously the resolver's MCP seam was unimplemented, so live MCP tool lists never reached the pushed snapshot. `agentcontext` is now **fully self-contained** for MCP: it connects to the MCP servers declared in the `.mcp.json` files its own watcher already discovers, lists their tools, and emits `KindMCPServer` resources. It does **not** depend on or modify `agent/x/agentmcp` — that package is left pristine and keeps serving the agent's MCP HTTP API. The two MCP paths run independently, which means the legacy package can be deleted later without touching this code. ## How - **Self-contained runner** (`agentcontext/mcprunner.go`): a one-shot MCP client (connect → initialize → list tools → close) with its own `.mcp.json` parser. A Manager goroutine (`runMCPSync`) reloads it whenever the discovered `KindMCPConfig` `path:contenthash` set changes, then re-resolves so the new tools are published. Per-server connects run in parallel (bounded) with a per-server timeout; a server that fails to connect is recorded as a failure rather than aborting the batch. Each connect also force-kills its subprocess on close, because mcp-go's stdio `Close()` closes stdin and then blocks on `cmd.Wait()` with no kill — a server that ignores stdin-close would otherwise stall the whole reload loop. - **Resource production** (`agentcontext/mcp.go`): `buildMCPServerResources` turns the runner's non-blocking per-server snapshot into `KindMCPServer` resources. Connected servers carry their sorted tools (`StatusOK`); failed servers surface as `StatusUnreadable` issues instead of vanishing; connected-but-no-tools-yet are skipped until a later reload. The content hash is tool-set sensitive. The resolver consumes this through a plain `MCPResources func() []Resource` field (no `MCPProvider` interface). - **Tool names**: emitted exactly as the server reports them. Flattening into a single namespace (e.g. `server__tool`) is left to the control plane in the next step, since each resource already carries the server name. - **Drift**: MCP resources are excluded from the snapshot aggregate/drift hash (`driftResources`). MCP servers connect asynchronously after boot; without this, a server finishing its connect would dirty every hydrated chat even though nothing the user pinned changed. - **Wiring** (`agent.go`): the manager is given `ManagerOptions.MCPExecer`/`MCPUpdateEnv`; `agent/x/agentmcp` is untouched. - **Config validation**: a structurally broken `.mcp.json` surfaces as `StatusInvalid` rather than silently dropping all its servers. coderd already persists `mcp_server`/`mcp_config` resource bodies (including tools), so no coderd or proto changes were required. ## Testing - **Unit**: `buildMCPServerResources` (grouping/sort/skip/failed/hash sensitivity), MCP resources applied via the resolver seam, MCP exclusion from the aggregate hash, `.mcp.json` parsing (transport inference, env expansion), `toolInputSchema`, and `mcpConfigSet` change detection. - **Proto serialization** (`TestDRPCPusher_HappyPathSerializesAllFields`): a `KindMCPServer` resource (tools + input schema) round-trips through `PushContextState` into the `MCPServerBody` wire form, asserting the server name, tool name/description, and the decoded `input_schema`. - **Manager-level, real subprocess** (`TestManager_MCPServerToolsInSnapshot`): a `.mcp.json` points at a re-exec'd fake stdio MCP server; the runner connects it and its `echo` tool surfaces as a `KindMCPServer` resource in the Manager snapshot — the same snapshot pushed to coderd — exercising `runMCPSync` and the resolver wiring end to end. - **Regression** (`TestManager_MCPServerHangingCloseDoesNotStall`): the fake server ignores stdin-close; the test asserts its tool still surfaces, proving the runner force-kills the subprocess instead of stalling the reload. Verified to fail without the fix. - All pass under `-race`; `go build ./...`, `go vet`, and `golangci-lint` are clean on the touched packages. ## Scope / follow-ups This is the agent-side production+push half. The chatd consumer (reading the pinned MCP resources for prompt/tool injection, including any server-prefix flattening of tool names) and removing the legacy `workspaceMCPToolsCache` pull path remain follow-ups, per the RFC rollout. While both `agent/x/agentmcp` and `agentcontext` exist, stdio MCP servers are spawned by both; this is intentional and temporary until `agentmcp` is removed. <details> <summary>Implementation plan and decisions</summary> **Goal:** produce live MCP server resources (with tools) from `agentcontext` and push them to coderd. **Starting state (main):** proto (`PushContextState`, `MCPServerBody`, `MCPTool`), the drpc adapter, coderd storage (`workspace_agent_context_resources`, body kind `mcp_server`), and the resolver's MCP seam already existed; nothing implemented the seam or fed live tools into the snapshot. **Decision (agentcontext fully separate from agentmcp):** `agentcontext` starts and lists its own MCP servers using only the connect-and-list half of an mcp-go client, driven by the `.mcp.json` files its existing watcher discovers. It shares no state with `agent/x/agentmcp` and does not import it. Two earlier revisions of this branch were discarded: (1) relocating `agentmcp` into `agentcontext` (rejected — it duplicates config parsing and file watching `agentcontext` already does); (2) reading `agentmcp`'s cached server snapshot via new accessors (rejected — unnecessary coupling between two packages that should simply run independently while one is being retired). The temporary double-spawn of stdio servers is the accepted cost of keeping the two paths cleanly separated until `agentmcp` is removed. **Decision (no tool-name prefixing, no MCPProvider interface):** the agent pushes raw, unflattened data — server name plus verbatim tool names — and lets the control plane own any `server__tool` flattening. With a single self-contained producer, the `MCPProvider` interface was collapsed into a `func() []Resource` field on the resolver. **Invariants held:** no secrets (env/headers) in pushed resources, only server/tool metadata; MCP excluded from the drift hash; the seam is non-blocking so the resolver never stalls on MCP I/O. </details> --- *This PR was created by Coder Agents on behalf of @kylecarbs.*	2026-06-21 16:31:05 -06:00
Kyle Carberry	992b1ffed1	feat(agent): serve context sources over the agent socket (#26526) ... ## Overview Split from #26466, scoped to **agent-only** changes. This PR exposes the agent's context sources and snapshots over the existing agent socket. There are no changes outside `agent/`. ## What's included - **agentsocket**: context source CRUD (`ContextSources`, `GetContextSource`, `AddContextSource`, `RemoveContextSource`) plus `GetContextSnapshot` and `ResyncContext` RPCs, with matching client methods and proto. The server receives the context `Manager` via `WithContextManager` and returns a clean error when it is absent. - **agentcontext**: the resync JSON response now carries the per-resource `Name`, keeping the HTTP resync payload in sync with the drpc `PushContextState` path in `agentsocket`. - **agent**: passes the context `Manager` to the socket server via `WithContextManager`. ## What's intentionally NOT here - No MCP wiring. There are no MCP additions in `agent.go` or `agentcontext`. MCP ownership will land later in `agentcontext`; this PR does not build on `agent/x/agentmcp`. - No changes to `agent/x/agentmcp` or the `agentcontext` resolver. The socket serves whatever context resources the `Manager` already resolves. <details> <summary>Context for reviewers</summary> This is one of several PRs split out of #26466. Earlier revisions also wired live MCP servers through the socket; that scope was removed so this PR stays purely socket + context plumbing inside `agent/`. The agentcontext resolver, `agent/x/agentmcp`, and `agent.go` MCP startup behavior are unchanged from `main`. </details> --- _Created by Coder Agents on behalf of @kylecarbs._	2026-06-18 13:00:28 -07:00
Kyle Carberry	b439b06ee6	feat: persist agent-pushed workspace context snapshots in coderd (#26145) ... Replaces the v2.10 `PushContextState` stub with a real coderd write path. Phase 1 of the chat-side persistence story; nothing reads these rows yet. Follows [#25983](https://github.com/coder/coder/pull/25983) and unblocks [CODAGT-569](https://linear.app/codercom/issue/CODAGT-569/enable-agent-api-v210-pushcontextstate-bump-currentminor-wire-coderd). ## What ships ### Schema (`000517_workspace_agent_context.{up,down}.sql`) Two new tables plus `api_key_scope` enum extensions: - `workspace_agent_context_snapshots` (PK `workspace_agent_id` to `workspace_agents(id) ON DELETE CASCADE`): one row per agent, overwritten per push. Holds `version`, `schema_version`, `aggregate_hash`, `snapshot_error`, `received_at`. - `workspace_agent_context_resources` (PK `(workspace_agent_id, source)`): per-resource state. `body_kind` and `status` are `TEXT` + `CHECK` so adding new wire kinds (the RFC's reserved PLUGIN/HOOK/SUBAGENT/COMMAND) is a one-line CHECK update plus a Go switch case. ### SQLC queries (`coderd/database/queries/workspaceagentcontext.sql`) - `UpsertWorkspaceAgentContextSnapshot` - `UpsertWorkspaceAgentContextResource` - `DeleteStaleWorkspaceAgentContextResources` (delete-where-source-not-in) - `GetLatestWorkspaceAgentContextSnapshot` - `ListWorkspaceAgentContextResources` ### Handler (`coderd/agentapi/context.go`) `ContextAPI` is a new sub-API. `PushContextState`: 1. Rejects `schema_version > 1` with a non-`Unimplemented` error so a forward-incompatible agent fails loudly during rollout instead of slipping into the permanent fallback path the `Unimplemented` translation reserves for old coderd deployments. 2. Validates resources: no empty/duplicate sources, every variant maps to a known body kind, every status maps to a known enum value, the `Body` oneof is set (even when status is non-OK, mirroring the wire guarantee so coderd can attribute failures to a known kind). 3. Inside `Database.InTx`, reads the existing snapshot. If the push is not `initial` and `version` is not strictly greater, returns `accepted = false` and leaves stored state untouched. Otherwise upserts the snapshot row, upserts each resource, then runs the stale-source prune so the snapshot and resource rows always agree. 4. Returns `accepted = true` on success. Resource bodies are stored as `protojson(body oneof variant)` in `body JSONB` with `body_kind` as the discriminator. Adding a new field to an existing variant is zero work since `protojson` tolerates new fields; adding a new variant is a CHECK + switch case. ### RBAC + dbauthz - New `ResourceWorkspaceAgentContext` (Create/Read/Update/Delete). - New `SubjectTypeAgentContext` plus `subjectAgentContext` system role and `dbauthz.AsAgentContext` helper. The push handler elevates to this subject; the agent's own role does not get direct write access to the table. - New `workspace_agent_context:*` API key scopes registered in the enum migration; internal-only (not added to `externalLowLevel`). ### Audit These rows are agent-pushed state, not user-authored. They are intentionally not added to `AuditActionMap` and not enumerated in `enterprise/audit/table.go`, matching `boundary_logs`, `workspace_agent_memory_resource_monitor`, etc. `enterprise/audit` tests pass unchanged. ## Tests - `coderd/agentapi/context_test.go`: 12 subtests covering accepts/rejects (schema version, empty/duplicate source, unknown status, missing body), version semantics (stale dropped, same-version replay dropped, `initial=true` overwrites lower version), variant coverage, non-OK status persistence, and the empty-active-set prune case. - `coderd/database/dbauthz/dbauthz_test.go`: 5 `MethodTestSuite` cases covering the new queries. - `coderd/rbac/roles_test.go`: `WorkspaceAgentContext` permission row asserting no human role currently has access. - `coderd/database/migrations/testdata/fixtures/000517_workspace_agent_context.up.sql`: one snapshot + one resource per known body kind plus a non-OK status, so the migration test suite never lands with these tables empty. ## Out of scope (later phases) - Chat hydration (`chats.context_aggregate_hash`, `last_injected_context`). - Dirty-bit fan-out and `PUT /chats/{id}/context`. - Agent-side `POST /api/v0/context/resync` barrier and the `coder exp chat context` CLI. - `codersdk` chat-context wire types and the dashboard Sources drawer. - Removal of the chatd per-turn pull fallback. ## Compat property This is a pure write path. If anything here returns errors the agent's `RunPush` loop backs off, no chat behavior changes, and the workspace keeps behaving exactly like it did before v2.10. <details> <summary>Implementation plan and decision log</summary> Key design calls: 1. **Concurrency**: Accept iff `req.Initial || req.Version > existing.Version`. The strict RFC reading ("version comparison is authoritative") locks restarted agents out because their per-process counter resets to 1; honoring `initial=true` reflects the real reboot reality while still rejecting steady-state replays/out-of-order pushes. 2. **Body encoding**: `protojson` over the oneof variant body proto, stored in JSONB with `body_kind` discriminator. Structured at the API/Go layer, schema-tolerant at the storage layer, and Phase 2 readers round-trip back via `protojson.Unmarshal`. 3. **Schema version rejection**: returns a normal error, not `Unimplemented`. The agent's `RunPush` loop only short-circuits on `Unimplemented`; that escape hatch is reserved for old coderd deployments. A forward-incompatible agent should retry-and-back-off, not flip the connection into permanent fallback. 4. **Validation strictness**: empty sources, duplicate sources, `STATUS_UNSPECIFIED`, and missing `Body` oneof variants are rejected before any write so a misbehaving agent cannot poison the snapshot table. Phase 2 readers can trust every row maps to a known proto variant. </details> _This PR was authored by Coder Agents on Kyle Carberry's behalf._	2026-06-15 09:38:52 -07:00
Kyle Carberry	cc9f10fbf2	fix(agent/agentcontext): canonicalize scan root in symlink boundary check (#26175)	2026-06-09 19:21:55 -07:00
Kyle Carberry	cd3692c0c2	feat: add agent-side workspace context sources and Agent API v2.10 PushContextState (#25983) ... Adds the agent half of the workspace context sources RFC. The agent now resolves instruction files, skills, and MCP configs into a typed `Snapshot`, watches the relevant paths recursively, exposes the source list over a workspace-agent HTTP API, and pushes each `Snapshot` to coderd over a new `PushContextState` RPC on Agent API v2.10. The coderd-side handler is a stub returning `Unimplemented` for now. Real persistence to `workspace_agent_context`, chatd hydration on dirty events, and the `KindMCPServer` MCP provider are tracked by [CODAGT-569](https://linear.app/codercom/issue/CODAGT-569/enable-agent-api-v210-pushcontextstate-bump-currentminor-wire-coderd). This matches the pattern used for v2.7 `ReportBoundaryLogs` in [#21293](https://github.com/coder/coder/pull/21293), which bumped the version and shipped a stub server so the wire and client could iterate before the persistence layer landed. ## What ships ### agent/agentcontext (new package) - `Source`, `Resource` (kinds `instruction_file`, `skill`, `mcp_config`, `mcp_server` plus reserved `plugin`/`hook`/`subagent`/`command`), `ResourceStatus`, `Snapshot`, `ComputeAggregateHash`. - `Manager` owns the in-memory source list, performs the initial resolve synchronously in `NewManager`, runs a re-resolve/watcher loop in `Run`, exposes `AddSource`/`RemoveSource`/`Sources`/`HasSource`/`Snapshot`/`SubscribeChanges`/`Resync`/`SeedSources`/`Close`. - `Resolver` walks scan roots, classifies recognized files, enforces 64 KiB per-resource, 2 MiB aggregate, and 500-resource caps with `StatusOversize`/`StatusExcluded`/`StatusUnreadable`/`StatusInvalid` outcomes, skips `node_modules`/`vendor`/etc., validates symlink targets stay inside the scan root, stamps `SourcePath` on user-derived resources, and optionally pulls MCP server tool lists via an `MCPProvider` interface. MCP config resources ship metadata only (size, hash) so secrets in env blocks never leave the agent. - `Watcher` is a recursive `fsnotify` wrapper with a 250 ms debounce, dynamic arming of newly created directories, and an ENOSPC-tolerant degraded mode that no-ops further syncs until the manager resyncs explicitly. - HTTP API for `GET/POST /sources`, `GET/DELETE /sources/{path}`, `POST /resync` mounted at `/api/v0/context`. - `Pusher` interface plus `RunPush` goroutine with exponential backoff capped at 30 s. `DRPCPusher` adapts the generated `DRPCAgentClient210` to `Pusher` and translates `drpcerr.Unimplemented` to `ErrPushUnimplemented` so the push loop exits cleanly when talking to coderd deployments that have not enabled the real handler. ### agent/proto (v2.10) - New messages `ContextResource`, `PushContextStateRequest`, `PushContextStateResponse` and the `PushContextState` RPC on `service Agent`. - Generated `DRPCAgentClient210` interface and `codersdk/agentsdk.Client.ConnectRPC210` / `ConnectRPC210WithRole`. - `tailnet/proto.CurrentMinor` bumped from `9` to `10`. ### Agent wiring - `agent.Options.Client` declares both v2.9 and v2.10 connectors; `run()` dials with `ConnectRPC210WithRole`. - `apiConnRoutineManager` holds a `DRPCAgentClient210`. Existing v2.8 routines keep their narrower `DRPCAgentClient28` signature thanks to interface embedding. - `startAgentAPI210` is the v2.10 counterpart to `startAgentAPI` for routines that need the new client. The push context state routine uses it. - A `contextManager` is constructed in `agent.init()`, seeded from the existing `CODER_AGENT_EXP_*_DIRS` env vars, started in its own goroutine under `gracefulCtx`, and closed in `agent.Close`. - `handleManifest` calls `Manager.SeedSources` for sources rooted at the manifest directory, then `Resync` after `manifest.Swap`, so the snapshot reflects the workspace working directory immediately instead of waiting for the next filesystem event. - HTTP routes mounted at `/api/v0/context` when the manager is up. ### Coderd stub `coderd/agentapi/context.go` returns `drpcerr.Unimplemented` for `PushContextState`. The real handler that persists `workspace_agent_context` rows, hydrates chats, and emits dirty events lives in CODAGT-569. ## Tests 24 tests across `agent/agentcontext` cover types, paths, resolver behavior with file caps, skill containers, MCP secret omission, symlink target validation, the recursive watcher firing on real fsnotify events, manager source CRUD / `Resync` / `SeedSources` / `Run` lifetime, the HTTP API, the DRPC adapter, and the push retry / initial-flag / unimplemented paths. Passes `go test -race -count=2`. `TestAgent_ContextStatePushed` boots a full agent against `agenttest.FakeAgentAPI` (which now records `PushContextState` traffic) and asserts the seeded `AGENTS.md` appears in a snapshot push with `schema_version = 1`. <details> <summary>Notes for reviewers</summary> - Source CRUD is workspace-agent-token only; coderd is not in the path for source mutation. - Per-resource cap 64 KiB, aggregate 2 MiB, count cap 500; resources past the cap ship with `StatusExcluded` and an empty payload so the aggregate hash still detects content edits. MCP-emitted resources enforce both a per-provider count cap and the aggregate byte cap. - Symlinks inside the scan root are followed; symlinks pointing outside (or broken) are rejected with `StatusExcluded` so credentials reachable via a stray symlink stay off the wire. - The initial push gates `lifecycle = ready` in the eventual full design. For this PR the `SeedSources` plus `handleManifest`-driven `Resync` keeps the snapshot fresh; the live push loop ships now and DRPCPusher translates the coderd `Unimplemented` stub into a clean exit. - The `PLUGIN`/`HOOK`/`SUBAGENT`/`COMMAND` kinds are reserved in proto and Go enums but unused; the Claude Code plugin resolver ships in a follow-up that does not need a schema migration. - Two follow-ups remain, both tracked by [CODAGT-569](https://linear.app/codercom/issue/CODAGT-569/enable-agent-api-v210-pushcontextstate-bump-currentminor-wire-coderd): (1) the chatd-side handler that persists snapshots and dirties chats; (2) the `coder exp chat context` CLI command set for `list`/`show`/`add`/`remove`/`refresh`. </details> _This PR was authored by Coder Agents on Kyle Carberry's behalf._	2026-06-08 12:08:40 -07:00