mirror of
https://github.com/coder/coder.git
synced 2026-09-24 15:04:27 +08:00
Adds the agent half of the workspace context sources RFC. The agent now resolves instruction files, skills, and MCP configs into a typed `Snapshot`, watches the relevant paths recursively, exposes the source list over a workspace-agent HTTP API, and pushes each `Snapshot` to coderd over a new `PushContextState` RPC on Agent API v2.10. The coderd-side handler is a stub returning `Unimplemented` for now. Real persistence to `workspace_agent_context`, chatd hydration on dirty events, and the `KindMCPServer` MCP provider are tracked by [CODAGT-569](https://linear.app/codercom/issue/CODAGT-569/enable-agent-api-v210-pushcontextstate-bump-currentminor-wire-coderd). This matches the pattern used for v2.7 `ReportBoundaryLogs` in [#21293](https://github.com/coder/coder/pull/21293), which bumped the version and shipped a stub server so the wire and client could iterate before the persistence layer landed. ## What ships ### agent/agentcontext (new package) - `Source`, `Resource` (kinds `instruction_file`, `skill`, `mcp_config`, `mcp_server` plus reserved `plugin`/`hook`/`subagent`/`command`), `ResourceStatus`, `Snapshot`, `ComputeAggregateHash`. - `Manager` owns the in-memory source list, performs the initial resolve synchronously in `NewManager`, runs a re-resolve/watcher loop in `Run`, exposes `AddSource`/`RemoveSource`/`Sources`/`HasSource`/`Snapshot`/`SubscribeChanges`/`Resync`/`SeedSources`/`Close`. - `Resolver` walks scan roots, classifies recognized files, enforces 64 KiB per-resource, 2 MiB aggregate, and 500-resource caps with `StatusOversize`/`StatusExcluded`/`StatusUnreadable`/`StatusInvalid` outcomes, skips `node_modules`/`vendor`/etc., validates symlink targets stay inside the scan root, stamps `SourcePath` on user-derived resources, and optionally pulls MCP server tool lists via an `MCPProvider` interface. MCP config resources ship metadata only (size, hash) so secrets in env blocks never leave the agent. - `Watcher` is a recursive `fsnotify` wrapper with a 250 ms debounce, dynamic arming of newly created directories, and an ENOSPC-tolerant degraded mode that no-ops further syncs until the manager resyncs explicitly. - HTTP API for `GET/POST /sources`, `GET/DELETE /sources/{path}`, `POST /resync` mounted at `/api/v0/context`. - `Pusher` interface plus `RunPush` goroutine with exponential backoff capped at 30 s. `DRPCPusher` adapts the generated `DRPCAgentClient210` to `Pusher` and translates `drpcerr.Unimplemented` to `ErrPushUnimplemented` so the push loop exits cleanly when talking to coderd deployments that have not enabled the real handler. ### agent/proto (v2.10) - New messages `ContextResource`, `PushContextStateRequest`, `PushContextStateResponse` and the `PushContextState` RPC on `service Agent`. - Generated `DRPCAgentClient210` interface and `codersdk/agentsdk.Client.ConnectRPC210` / `ConnectRPC210WithRole`. - `tailnet/proto.CurrentMinor` bumped from `9` to `10`. ### Agent wiring - `agent.Options.Client` declares both v2.9 and v2.10 connectors; `run()` dials with `ConnectRPC210WithRole`. - `apiConnRoutineManager` holds a `DRPCAgentClient210`. Existing v2.8 routines keep their narrower `DRPCAgentClient28` signature thanks to interface embedding. - `startAgentAPI210` is the v2.10 counterpart to `startAgentAPI` for routines that need the new client. The push context state routine uses it. - A `contextManager` is constructed in `agent.init()`, seeded from the existing `CODER_AGENT_EXP_*_DIRS` env vars, started in its own goroutine under `gracefulCtx`, and closed in `agent.Close`. - `handleManifest` calls `Manager.SeedSources` for sources rooted at the manifest directory, then `Resync` after `manifest.Swap`, so the snapshot reflects the workspace working directory immediately instead of waiting for the next filesystem event. - HTTP routes mounted at `/api/v0/context` when the manager is up. ### Coderd stub `coderd/agentapi/context.go` returns `drpcerr.Unimplemented` for `PushContextState`. The real handler that persists `workspace_agent_context` rows, hydrates chats, and emits dirty events lives in CODAGT-569. ## Tests 24 tests across `agent/agentcontext` cover types, paths, resolver behavior with file caps, skill containers, MCP secret omission, symlink target validation, the recursive watcher firing on real fsnotify events, manager source CRUD / `Resync` / `SeedSources` / `Run` lifetime, the HTTP API, the DRPC adapter, and the push retry / initial-flag / unimplemented paths. Passes `go test -race -count=2`. `TestAgent_ContextStatePushed` boots a full agent against `agenttest.FakeAgentAPI` (which now records `PushContextState` traffic) and asserts the seeded `AGENTS.md` appears in a snapshot push with `schema_version = 1`. <details> <summary>Notes for reviewers</summary> - Source CRUD is workspace-agent-token only; coderd is not in the path for source mutation. - Per-resource cap 64 KiB, aggregate 2 MiB, count cap 500; resources past the cap ship with `StatusExcluded` and an empty payload so the aggregate hash still detects content edits. MCP-emitted resources enforce both a per-provider count cap and the aggregate byte cap. - Symlinks inside the scan root are followed; symlinks pointing outside (or broken) are rejected with `StatusExcluded` so credentials reachable via a stray symlink stay off the wire. - The initial push gates `lifecycle = ready` in the eventual full design. For this PR the `SeedSources` plus `handleManifest`-driven `Resync` keeps the snapshot fresh; the live push loop ships now and DRPCPusher translates the coderd `Unimplemented` stub into a clean exit. - The `PLUGIN`/`HOOK`/`SUBAGENT`/`COMMAND` kinds are reserved in proto and Go enums but unused; the Claude Code plugin resolver ships in a follow-up that does not need a schema migration. - Two follow-ups remain, both tracked by [CODAGT-569](https://linear.app/codercom/issue/CODAGT-569/enable-agent-api-v210-pushcontextstate-bump-currentminor-wire-coderd): (1) the chatd-side handler that persists snapshots and dirties chats; (2) the `coder exp chat context` CLI command set for `list`/`show`/`add`/`remove`/`refresh`. </details> _This PR was authored by Coder Agents on Kyle Carberry's behalf._
399 lines
11 KiB
Go
399 lines
11 KiB
Go
package agentcontext_test
|
|
|
|
import (
|
|
"context"
|
|
"os"
|
|
"path/filepath"
|
|
"runtime"
|
|
"sync/atomic"
|
|
"testing"
|
|
"time"
|
|
|
|
"github.com/stretchr/testify/require"
|
|
|
|
"github.com/coder/coder/v2/agent/agentcontext"
|
|
"github.com/coder/coder/v2/testutil"
|
|
)
|
|
|
|
// TestMain points the test binary's HOME (and USERPROFILE on
|
|
// Windows) at a fresh empty directory before any test runs.
|
|
// The package's built-in scan roots (~/.coder,
|
|
// ~/.coder/skills, ~/.claude/plugins/cache) canonicalize
|
|
// against this directory, so they resolve to non-existent
|
|
// paths and the resolver silently skips them. Without this,
|
|
// running the tests on a developer host pulls real Coder and
|
|
// Claude config files into snapshots and breaks every
|
|
// Len(Resources, N) assertion.
|
|
func TestMain(m *testing.M) {
|
|
home, err := os.MkdirTemp("", "agentcontext-test-home-")
|
|
if err != nil {
|
|
panic(err)
|
|
}
|
|
if err := os.Setenv("HOME", home); err != nil {
|
|
panic(err)
|
|
}
|
|
if runtime.GOOS == "windows" {
|
|
if err := os.Setenv("USERPROFILE", home); err != nil {
|
|
panic(err)
|
|
}
|
|
}
|
|
code := m.Run()
|
|
_ = os.RemoveAll(home)
|
|
os.Exit(code)
|
|
}
|
|
|
|
func newTestManager(t *testing.T, opts agentcontext.ManagerOptions) *agentcontext.Manager {
|
|
t.Helper()
|
|
opts.Logger = testutil.Logger(t).Named("agentcontext-test")
|
|
m := agentcontext.NewManager(opts)
|
|
t.Cleanup(func() { _ = m.Close() })
|
|
return m
|
|
}
|
|
|
|
func TestManager_InitialSnapshotIsPopulated(t *testing.T) {
|
|
t.Parallel()
|
|
dir := t.TempDir()
|
|
mustWriteFile(t, filepath.Join(dir, "AGENTS.md"), "boot snapshot")
|
|
|
|
m := newTestManager(t, agentcontext.ManagerOptions{
|
|
WorkingDir: func() string { return dir },
|
|
})
|
|
|
|
snap := m.Snapshot()
|
|
require.Equal(t, uint64(1), snap.Version)
|
|
require.Equal(t, agentcontext.CurrentSchemaVersion, snap.SchemaVersion)
|
|
require.Len(t, snap.Resources, 1)
|
|
}
|
|
|
|
func TestManager_AddSourceTriggersResolve(t *testing.T) {
|
|
t.Parallel()
|
|
wd := t.TempDir()
|
|
src := t.TempDir()
|
|
mustWriteFile(t, filepath.Join(src, "AGENTS.md"), "from source")
|
|
|
|
m := newTestManager(t, agentcontext.ManagerOptions{
|
|
WorkingDir: func() string { return wd },
|
|
AllowedRoots: []string{wd, src},
|
|
})
|
|
|
|
ctx := testutil.Context(t, testutil.WaitLong)
|
|
go func() { _ = m.Run(ctx) }()
|
|
|
|
t.Cleanup(func() { _ = m.Close() })
|
|
|
|
// Subscribe before mutating so we observe the broadcast.
|
|
ch, unsub := m.SubscribeChanges()
|
|
defer unsub()
|
|
|
|
added, err := m.AddSource(agentcontext.Source{Path: src})
|
|
require.NoError(t, err)
|
|
require.Equal(t, src, added.Path)
|
|
|
|
select {
|
|
case <-ch:
|
|
case <-time.After(testutil.WaitShort):
|
|
t.Fatalf("expected a change broadcast after AddSource")
|
|
}
|
|
|
|
snap := m.Snapshot()
|
|
require.Greater(t, snap.Version, uint64(1))
|
|
|
|
found := false
|
|
for _, r := range snap.Resources {
|
|
if r.Kind == agentcontext.KindInstructionFile && r.SourcePath == src {
|
|
found = true
|
|
}
|
|
}
|
|
require.True(t, found, "expected AGENTS.md attributed to the user source")
|
|
}
|
|
|
|
func TestManager_AddSourceRejectsOutsideAllowedRoots(t *testing.T) {
|
|
t.Parallel()
|
|
wd := t.TempDir()
|
|
outside := t.TempDir()
|
|
|
|
m := newTestManager(t, agentcontext.ManagerOptions{
|
|
WorkingDir: func() string { return wd },
|
|
AllowedRoots: []string{wd},
|
|
})
|
|
|
|
_, err := m.AddSource(agentcontext.Source{Path: outside})
|
|
require.Error(t, err)
|
|
}
|
|
|
|
// TestManager_AddSourceAcceptsLateWorkingDir mirrors the agent's
|
|
// real boot order: AllowedRoots is configured before the
|
|
// manifest provides the workspace working directory. The Manager
|
|
// must consult WorkingDir on every check so paths under the
|
|
// resolved working dir validate once the manifest lands.
|
|
func TestManager_AddSourceAcceptsLateWorkingDir(t *testing.T) {
|
|
t.Parallel()
|
|
wd := t.TempDir()
|
|
var resolved atomic.Pointer[string]
|
|
m := newTestManager(t, agentcontext.ManagerOptions{
|
|
WorkingDir: func() string {
|
|
if p := resolved.Load(); p != nil {
|
|
return *p
|
|
}
|
|
return ""
|
|
},
|
|
AllowedRoots: []string{"/never-used-home"},
|
|
})
|
|
|
|
// Before the manifest "loads", workingDir is empty; sources
|
|
// under wd must be rejected.
|
|
_, err := m.AddSource(agentcontext.Source{Path: wd})
|
|
require.Error(t, err)
|
|
|
|
// After the manifest "loads", workingDir resolves and the
|
|
// same path validates without restarting the Manager.
|
|
resolved.Store(&wd)
|
|
_, err = m.AddSource(agentcontext.Source{Path: wd})
|
|
require.NoError(t, err)
|
|
}
|
|
|
|
func TestManager_AddSourceIsIdempotent(t *testing.T) {
|
|
t.Parallel()
|
|
wd := t.TempDir()
|
|
src := t.TempDir()
|
|
|
|
m := newTestManager(t, agentcontext.ManagerOptions{
|
|
WorkingDir: func() string { return wd },
|
|
AllowedRoots: []string{wd, src},
|
|
})
|
|
|
|
added1, err := m.AddSource(agentcontext.Source{Path: src})
|
|
require.NoError(t, err)
|
|
added2, err := m.AddSource(agentcontext.Source{Path: src})
|
|
require.NoError(t, err)
|
|
require.Equal(t, added1.Path, added2.Path)
|
|
|
|
sources := m.Sources()
|
|
require.Len(t, sources, 1)
|
|
}
|
|
|
|
func TestManager_RemoveSource(t *testing.T) {
|
|
t.Parallel()
|
|
wd := t.TempDir()
|
|
src := t.TempDir()
|
|
|
|
m := newTestManager(t, agentcontext.ManagerOptions{
|
|
WorkingDir: func() string { return wd },
|
|
AllowedRoots: []string{wd, src},
|
|
})
|
|
|
|
_, err := m.AddSource(agentcontext.Source{Path: src})
|
|
require.NoError(t, err)
|
|
require.NoError(t, m.RemoveSource(src))
|
|
require.Empty(t, m.Sources())
|
|
|
|
err = m.RemoveSource(src)
|
|
require.ErrorIs(t, err, agentcontext.ErrSourceNotFound)
|
|
}
|
|
|
|
func TestManager_HasSource(t *testing.T) {
|
|
t.Parallel()
|
|
wd := t.TempDir()
|
|
src := t.TempDir()
|
|
|
|
m := newTestManager(t, agentcontext.ManagerOptions{
|
|
WorkingDir: func() string { return wd },
|
|
AllowedRoots: []string{wd, src},
|
|
})
|
|
|
|
canonical, ok := m.HasSource(src)
|
|
require.False(t, ok)
|
|
require.Equal(t, src, canonical)
|
|
|
|
_, err := m.AddSource(agentcontext.Source{Path: src})
|
|
require.NoError(t, err)
|
|
|
|
canonical, ok = m.HasSource(src)
|
|
require.True(t, ok)
|
|
require.Equal(t, src, canonical)
|
|
}
|
|
|
|
func TestManager_ResyncReturnsLatestSnapshot(t *testing.T) {
|
|
t.Parallel()
|
|
wd := t.TempDir()
|
|
mustWriteFile(t, filepath.Join(wd, "AGENTS.md"), "first")
|
|
|
|
m := newTestManager(t, agentcontext.ManagerOptions{
|
|
WorkingDir: func() string { return wd },
|
|
})
|
|
|
|
ctx := testutil.Context(t, testutil.WaitLong)
|
|
runDone := make(chan struct{})
|
|
go func() {
|
|
defer close(runDone)
|
|
_ = m.Run(ctx)
|
|
}()
|
|
t.Cleanup(func() {
|
|
_ = m.Close()
|
|
<-runDone
|
|
})
|
|
|
|
// Mutate AGENTS.md and call Resync. The returned
|
|
// snapshot must reflect the new content.
|
|
require.NoError(t, os.WriteFile(filepath.Join(wd, "AGENTS.md"), []byte("second content edit"), 0o600))
|
|
|
|
snap, err := m.Resync(ctx)
|
|
require.NoError(t, err)
|
|
|
|
require.Len(t, snap.Resources, 1)
|
|
require.Equal(t, "second content edit", string(snap.Resources[0].Payload))
|
|
}
|
|
|
|
// TestManager_ResyncCanceledKeepsLiveSnapshot guards CRF-44:
|
|
// a context cancellation mid-walk must not replace the live
|
|
// Snapshot with an empty one. Resync returns the existing
|
|
// Snapshot and ctx.Err() instead of publishing a stub.
|
|
func TestManager_ResyncCanceledKeepsLiveSnapshot(t *testing.T) {
|
|
t.Parallel()
|
|
wd := t.TempDir()
|
|
mustWriteFile(t, filepath.Join(wd, "AGENTS.md"), "live content")
|
|
|
|
m := newTestManager(t, agentcontext.ManagerOptions{
|
|
WorkingDir: func() string { return wd },
|
|
})
|
|
|
|
// Capture the live snapshot the Manager populated at
|
|
// construction time.
|
|
live := m.Snapshot()
|
|
require.Len(t, live.Resources, 1)
|
|
require.Equal(t, "live content", string(live.Resources[0].Payload))
|
|
|
|
// Cancel the context before calling Resync so
|
|
// ResolveContext observes the cancellation.
|
|
ctx, cancel := context.WithCancel(context.Background())
|
|
cancel()
|
|
|
|
snap, err := m.Resync(ctx)
|
|
require.ErrorIs(t, err, context.Canceled)
|
|
// The returned snapshot must still expose the live
|
|
// resources, not an empty result from the canceled walk.
|
|
require.Len(t, snap.Resources, 1)
|
|
require.Equal(t, "live content", string(snap.Resources[0].Payload))
|
|
|
|
// The next Snapshot call must also return live content;
|
|
// no stub was published.
|
|
after := m.Snapshot()
|
|
require.Equal(t, live.Version, after.Version)
|
|
require.Len(t, after.Resources, 1)
|
|
}
|
|
|
|
func TestManager_InitialSourcesSeeded(t *testing.T) {
|
|
t.Parallel()
|
|
wd := t.TempDir()
|
|
src := t.TempDir()
|
|
mustWriteFile(t, filepath.Join(src, "AGENTS.md"), "from initial")
|
|
|
|
m := newTestManager(t, agentcontext.ManagerOptions{
|
|
WorkingDir: func() string { return wd },
|
|
AllowedRoots: []string{wd, src},
|
|
InitialSources: []agentcontext.Source{{Path: src}},
|
|
})
|
|
|
|
sources := m.Sources()
|
|
require.Len(t, sources, 1)
|
|
require.Equal(t, src, sources[0].Path)
|
|
|
|
snap := m.Snapshot()
|
|
require.Len(t, snap.Resources, 1)
|
|
require.Equal(t, src, snap.Resources[0].SourcePath)
|
|
}
|
|
|
|
// TestManager_SeedSourcesLateBindsAfterManifest models the
|
|
// agent's behavior when CODER_AGENT_EXP_*_DIRS contains a
|
|
// relative path that cannot resolve until the manifest's
|
|
// working directory lands. SeedSources must adopt the
|
|
// previously-unresolvable path, bypass AllowedRoots
|
|
// validation, and trigger a re-resolve.
|
|
func TestManager_SeedSourcesLateBindsAfterManifest(t *testing.T) {
|
|
t.Parallel()
|
|
wd := t.TempDir()
|
|
late := t.TempDir()
|
|
mustWriteFile(t, filepath.Join(late, "AGENTS.md"), "late binding")
|
|
|
|
// AllowedRoots intentionally omits `late` so AddSource
|
|
// would reject it. SeedSources must accept it anyway,
|
|
// since the path comes from the trusted template config.
|
|
m := newTestManager(t, agentcontext.ManagerOptions{
|
|
WorkingDir: func() string { return wd },
|
|
AllowedRoots: []string{wd},
|
|
})
|
|
|
|
require.Empty(t, m.Sources())
|
|
|
|
m.SeedSources([]agentcontext.Source{{Path: late}})
|
|
|
|
sources := m.Sources()
|
|
require.Len(t, sources, 1)
|
|
require.Equal(t, late, sources[0].Path)
|
|
|
|
snap, err := m.Resync(testutil.Context(t, testutil.WaitShort))
|
|
require.NoError(t, err)
|
|
require.Len(t, snap.Resources, 1)
|
|
require.Equal(t, late, snap.Resources[0].SourcePath)
|
|
}
|
|
|
|
func TestManager_CloseIsIdempotent(t *testing.T) {
|
|
t.Parallel()
|
|
m := newTestManager(t, agentcontext.ManagerOptions{
|
|
WorkingDir: func() string { return t.TempDir() },
|
|
})
|
|
require.NoError(t, m.Close())
|
|
require.NoError(t, m.Close())
|
|
}
|
|
|
|
func TestManager_RunOnce(t *testing.T) {
|
|
t.Parallel()
|
|
m := newTestManager(t, agentcontext.ManagerOptions{
|
|
WorkingDir: func() string { return t.TempDir() },
|
|
})
|
|
ctx, cancel := context.WithCancel(testutil.Context(t, testutil.WaitShort))
|
|
defer cancel()
|
|
go func() { _ = m.Run(ctx) }()
|
|
|
|
// Wait for Run to claim the running flag, then verify the
|
|
// second call rejects with a deterministic error rather than
|
|
// racing the scheduler.
|
|
select {
|
|
case <-agentcontext.ManagerStarted(m):
|
|
case <-ctx.Done():
|
|
t.Fatalf("manager never started: %v", ctx.Err())
|
|
}
|
|
|
|
err := m.Run(ctx)
|
|
require.Error(t, err)
|
|
require.Contains(t, err.Error(), "more than once")
|
|
cancel()
|
|
_ = m.Close()
|
|
}
|
|
|
|
func TestManager_SubscribeBroadcastOnChange(t *testing.T) {
|
|
t.Parallel()
|
|
wd := t.TempDir()
|
|
src := t.TempDir()
|
|
|
|
m := newTestManager(t, agentcontext.ManagerOptions{
|
|
WorkingDir: func() string { return wd },
|
|
AllowedRoots: []string{wd, src},
|
|
})
|
|
|
|
ctx := testutil.Context(t, testutil.WaitLong)
|
|
go func() { _ = m.Run(ctx) }()
|
|
|
|
ch, unsub := m.SubscribeChanges()
|
|
defer unsub()
|
|
|
|
_, err := m.AddSource(agentcontext.Source{Path: src})
|
|
require.NoError(t, err)
|
|
|
|
select {
|
|
case <-ch:
|
|
case <-time.After(testutil.WaitShort):
|
|
t.Fatal("expected subscriber to be notified")
|
|
}
|
|
}
|