Files
Thomas Kosiewski 9aa7130e01 feat: evaluate experiment rules per user in discovery and chatd (#29964)
Slice 3 of 4 for runtime experiment rules: evaluate stored rules per
user in discovery, the frontend experiments query, and the pilot
`mcp-tool-search` gate in chatd. There is still no way to edit rules
through the product; that comes in slice 4.

With no rules stored and a healthy database, backend decisions match
today. This PR deliberately changes three things:

1. **Fail closed on rule reads.** If the rules read fails,
`mcp-tool-search` is off, even when it is enabled at startup.
2. **Frontend freshness.** The experiments query now refetches.
3. **Caching.** `/api/v2/experiments` and rendered HTML are sent with
`Cache-Control: no-store`. Static assets keep their long-lived caching.

## What's in it

- **coderd** builds `api.ExperimentEvaluator` from
`experiments.NewDBStore` plus the startup `--experiments` list.
- **Discovery:** `GET /api/v2/experiments` and the `<meta
property="experiments">` tag both use `EnabledExperiments(ctx, caller)`.
- **chatd:** `mcp-tool-search` is now user-scoped.
`chatd.Config.ExperimentEvaluator` is required, so `chatd.New` returns
an error without it. `prepareGeneration` decides once per turn for the
chat owner (`chat.OwnerID`); child chats inherit the owner.
- **Ruleguard** (`userScopedExperimentStaticCheck` in
`scripts/rules.go`): rejects static `Experiments.Enabled(...)` and
`httpmw.RequireExperiment*(...)` checks of user-scoped experiments in
server packages. A codersdk test keeps its list equal to
`codersdk.ExperimentsUserScoped`.
- **Frontend:** the experiments query is keyed by user ID and seeded
from the meta tag only when the embedded user matches. It is stale 60 s
after page load and refetches on mount, window focus and reconnect.
Logout also clears the experiments metadata. The dashboard keeps the
last loaded list when a background refetch fails, instead of replacing
the page with an error.
- **Once per turn:** chatd decides `mcp-tool-search` on a turn's first
step and reuses that decision for the turn's later steps, so a rule
change applies from the next turn and never withdraws a `find_tools`
call the model already made. The decision lives on the chat runner,
keyed by the turn's prompt row, like the stop-nudge tracker. A runner
handoff in the middle of a turn evaluates again.
- **Reference audit:** the only non-test server check of
`mcp-tool-search` is in `generation_preparer.go`, and it now goes
through the evaluator.

## Testing

- **Go:** per-user results and `no-store` on `/api/v2/experiments`; the
per-user meta tag and the HTML cache headers in `site`; group and org
conditions end to end in enterprise; chatd owner eligibility, on → off →
on across turns including a child chat, and a failed rules read.
- **Vitest:** user-keyed seeding, staleness from page load after a
remount, refetch on focus, logout clearing, and the dashboard surviving
a failed refetch.

Remote dogfood UAT through Coder Agents passed on a licensed test
deployment:
- per-user discovery in both the API and the meta tag;
- widening a rule, and turning it off, with no restart;
- `off` beating a startup default, and `inherit` restoring it;
- fail closed on malformed JSON and on an unknown mode, with no
condition text in the logs;
- the `no-store` headers;
- real-model chatd enforcement: `find_tools` offered on turn 1, gone
after `off`, back on turn 3.

That round tested `b57207d4ee`. The later commits change only tests, and
a follow-up round will cover the final head.

**For the author:** `coderd/x/chatd/ARCHITECTURE.md`'s list of the
runner's local state doesn't mention the per-turn trackers (debug turn,
session start, stop nudges, and now this experiment decision). I did not
edit that document.

**Known behavior:** a broken rule logs one error line per evaluation
(per request or turn), with only the experiment, revision, category and
position.

## Stack

1. #29869: evaluator library
2. #29898: storage and audit types
3. **This PR:** enforcement and discovery
4. Control plane (experimental API, `coder exp experiment-rules` CLI,
docs)

> [!NOTE]
> This PR was written by an AI agent (Xum).

---
_Generated with [`xum`](https://github.com/coder/xum) • Model:
`anthropic:claude-opus-5-5` • Thinking: `high`_

---------

Signed-off-by: Thomas Kosiewski <tk@coder.com>
2026-09-29 12:23:00 +00:00

1214 lines
36 KiB
Go

package site_test
import (
"bytes"
"context"
"encoding/json"
"fmt"
"html"
"io"
"io/fs"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strconv"
"strings"
"sync/atomic"
"testing"
"testing/fstest"
"time"
"github.com/go-chi/chi/v5"
"github.com/go-chi/chi/v5/middleware"
"github.com/google/uuid"
"github.com/prometheus/client_golang/prometheus"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"golang.org/x/exp/maps"
"github.com/coder/coder/v2/coderd/appearance"
"github.com/coder/coder/v2/coderd/coderdtest"
"github.com/coder/coder/v2/coderd/database"
"github.com/coder/coder/v2/coderd/database/db2sdk"
"github.com/coder/coder/v2/coderd/database/dbauthz"
"github.com/coder/coder/v2/coderd/database/dbgen"
"github.com/coder/coder/v2/coderd/database/dbtestutil"
"github.com/coder/coder/v2/coderd/database/dbtime"
"github.com/coder/coder/v2/coderd/experiments"
"github.com/coder/coder/v2/coderd/httpmw"
"github.com/coder/coder/v2/coderd/rbac"
"github.com/coder/coder/v2/coderd/telemetry"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/site"
"github.com/coder/coder/v2/testutil"
)
type staticAppearanceFetcher struct {
cfg codersdk.AppearanceConfig
}
func (f staticAppearanceFetcher) Fetch(context.Context) (codersdk.AppearanceConfig, error) {
return f.cfg, nil
}
func TestInjectionAppearanceEscapesMetaAttributes(t *testing.T) {
t.Parallel()
const (
applicationName = `Coder"><script>alert(1)</script>`
logoURL = `https://example.com/logo.png"><img src=x onerror=alert(1)>`
)
tests := []struct {
name string
authenticated bool
}{
{
name: "unauthenticated",
},
{
name: "authenticated",
authenticated: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
siteFS := fstest.MapFS{
"index.html": &fstest.MapFile{
Data: []byte(`<meta name="application-name" content="{{ .ApplicationName }}" /><meta property="logo-url" content="{{ .LogoURL }}" />`),
},
}
db, _ := dbtestutil.NewDB(t)
var appearanceFetcher atomic.Pointer[appearance.Fetcher]
fetcher := appearance.Fetcher(staticAppearanceFetcher{cfg: codersdk.AppearanceConfig{
ApplicationName: applicationName,
LogoURL: logoURL,
}})
appearanceFetcher.Store(&fetcher)
handler, err := site.New(&site.Options{
Telemetry: telemetry.NewNoop(),
Database: db,
SiteFS: siteFS,
AppearanceFetcher: &appearanceFetcher,
})
require.NoError(t, err)
r := httptest.NewRequest("GET", "/", nil)
if tt.authenticated {
user := dbgen.User(t, db, database.User{})
_, token := dbgen.APIKey(t, db, database.APIKey{
UserID: user.ID,
ExpiresAt: time.Now().Add(time.Hour),
})
r.Header.Set(codersdk.SessionTokenHeader, token)
}
rw := httptest.NewRecorder()
handler.ServeHTTP(rw, r)
require.Equal(t, http.StatusOK, rw.Code)
body := rw.Body.String()
require.True(t, strings.Contains(body, html.EscapeString(applicationName)), "application name must be HTML escaped")
require.True(t, strings.Contains(body, html.EscapeString(logoURL)), "logo URL must be HTML escaped")
require.False(t, strings.Contains(body, applicationName), "raw application name must not be rendered")
require.False(t, strings.Contains(body, logoURL), "raw logo URL must not be rendered")
})
}
}
func TestInjection(t *testing.T) {
t.Parallel()
siteFS := fstest.MapFS{
"index.html": &fstest.MapFile{
Data: []byte("{{ .User }}"),
},
}
db, _ := dbtestutil.NewDB(t)
handler, err := site.New(&site.Options{
Telemetry: telemetry.NewNoop(),
Database: db,
SiteFS: siteFS,
})
require.NoError(t, err)
user := dbgen.User(t, db, database.User{})
_, token := dbgen.APIKey(t, db, database.APIKey{
UserID: user.ID,
ExpiresAt: time.Now().Add(time.Hour),
})
r := httptest.NewRequest("GET", "/", nil)
r.Header.Set(codersdk.SessionTokenHeader, token)
rw := httptest.NewRecorder()
handler.ServeHTTP(rw, r)
require.Equal(t, http.StatusOK, rw.Code)
var got codersdk.User
err = json.Unmarshal([]byte(html.UnescapeString(rw.Body.String())), &got)
require.NoError(t, err)
// This will update as part of the request!
got.LastSeenAt = user.LastSeenAt
// json.Unmarshal doesn't parse the timezone correctly
got.CreatedAt = got.CreatedAt.In(user.CreatedAt.Location())
got.UpdatedAt = got.UpdatedAt.In(user.CreatedAt.Location())
require.Equal(t, db2sdk.User(user, []uuid.UUID{}), got)
}
func TestInjectionUserAppearance(t *testing.T) {
t.Parallel()
siteFS := fstest.MapFS{
"index.html": &fstest.MapFile{
Data: []byte("{{ .UserAppearance }}"),
},
}
db, _ := dbtestutil.NewDB(t)
handler, err := site.New(&site.Options{
Telemetry: telemetry.NewNoop(),
Database: db,
SiteFS: siteFS,
})
require.NoError(t, err)
user := dbgen.User(t, db, database.User{})
ctx := context.Background()
_, err = db.UpdateUserThemePreference(ctx, database.UpdateUserThemePreferenceParams{
UserID: user.ID,
ThemePreference: "dark-tritan",
})
require.NoError(t, err)
_, err = db.UpdateUserThemeMode(ctx, database.UpdateUserThemeModeParams{
UserID: user.ID,
ThemeMode: string(codersdk.ThemeModeSync),
})
require.NoError(t, err)
_, err = db.UpdateUserThemeLight(ctx, database.UpdateUserThemeLightParams{
UserID: user.ID,
ThemeLight: "light-tritan",
})
require.NoError(t, err)
_, err = db.UpdateUserThemeDark(ctx, database.UpdateUserThemeDarkParams{
UserID: user.ID,
ThemeDark: "dark-tritan",
})
require.NoError(t, err)
_, err = db.UpdateUserTerminalFont(ctx, database.UpdateUserTerminalFontParams{
UserID: user.ID,
TerminalFont: string(codersdk.TerminalFontFiraCode),
})
require.NoError(t, err)
_, token := dbgen.APIKey(t, db, database.APIKey{
UserID: user.ID,
ExpiresAt: time.Now().Add(time.Hour),
})
r := httptest.NewRequest("GET", "/", nil)
r.Header.Set(codersdk.SessionTokenHeader, token)
rw := httptest.NewRecorder()
handler.ServeHTTP(rw, r)
require.Equal(t, http.StatusOK, rw.Code)
var got codersdk.UserAppearanceSettings
err = json.Unmarshal([]byte(html.UnescapeString(rw.Body.String())), &got)
require.NoError(t, err)
require.Equal(t, codersdk.UserAppearanceSettings{
ThemePreference: "dark-tritan",
ThemeMode: codersdk.ThemeModeSync,
ThemeLight: "light-tritan",
ThemeDark: "dark-tritan",
TerminalFont: codersdk.TerminalFontFiraCode,
}, got)
}
func TestRenderPermissionsResolvesMe(t *testing.T) {
t.Parallel()
// GIVEN: a site handler wired to a real RBAC authorizer and a
// template that renders only the SSR permissions JSON.
siteFS := fstest.MapFS{
"index.html": &fstest.MapFile{
Data: []byte("{{ .Permissions }}"),
},
}
db, _ := dbtestutil.NewDB(t)
authorizer := rbac.NewStrictCachingAuthorizer(prometheus.NewRegistry())
handler, err := site.New(&site.Options{
Telemetry: telemetry.NewNoop(),
Database: db,
SiteFS: siteFS,
Authorizer: authorizer,
})
require.NoError(t, err)
org := dbgen.Organization(t, db, database.Organization{})
orgMember := dbgen.User(t, db, database.User{})
dbgen.OrganizationMember(t, db, database.OrganizationMember{
OrganizationID: org.ID,
UserID: orgMember.ID,
})
_, orgMemberToken := dbgen.APIKey(t, db, database.APIKey{
UserID: orgMember.ID,
ExpiresAt: time.Now().Add(time.Hour),
})
// WHEN: the user loads the page.
r := httptest.NewRequest("GET", "/", nil)
r.Header.Set(codersdk.SessionTokenHeader, orgMemberToken)
rw := httptest.NewRecorder()
handler.ServeHTTP(rw, r)
require.Equal(t, http.StatusOK, rw.Code)
// The any_org check finds chat permission through the user's membership.
var memberPerms codersdk.AuthorizationResponse
err = json.Unmarshal([]byte(html.UnescapeString(rw.Body.String())), &memberPerms)
require.NoError(t, err)
assert.True(t, memberPerms["createChat"], "organization member should have createChat = true")
// THEN: createWorkspace = true because the organization-member role
// grants creating a workspace owned by the member, and owner_id "me"
// resolves to the requesting user.
assert.True(t, memberPerms["createWorkspace"], "org member should have createWorkspace = true")
userWithoutMembership := dbgen.User(t, db, database.User{})
_, tokenWithoutMembership := dbgen.APIKey(t, db, database.APIKey{
UserID: userWithoutMembership.ID,
ExpiresAt: time.Now().Add(time.Hour),
})
// WHEN: the user loads the page.
r = httptest.NewRequest("GET", "/", nil)
r.Header.Set(codersdk.SessionTokenHeader, tokenWithoutMembership)
rw = httptest.NewRecorder()
handler.ServeHTTP(rw, r)
require.Equal(t, http.StatusOK, rw.Code)
var permsWithoutMembership codersdk.AuthorizationResponse
err = json.Unmarshal([]byte(html.UnescapeString(rw.Body.String())), &permsWithoutMembership)
require.NoError(t, err)
assert.False(t, permsWithoutMembership["createChat"], "user without an organization membership should have createChat = false")
// THEN: createWorkspace = false because the user belongs to no
// organization, so the any_org check has no memberships to satisfy it.
assert.False(t, permsWithoutMembership["createWorkspace"], "user without an org membership should have createWorkspace = false")
// GIVEN: an org member whose only membership carries the
// workspace-creation ban role.
bannedUser := dbgen.User(t, db, database.User{})
dbgen.OrganizationMember(t, db, database.OrganizationMember{
OrganizationID: org.ID,
UserID: bannedUser.ID,
Roles: []string{rbac.RoleOrgWorkspaceCreationBan()},
})
_, bannedToken := dbgen.APIKey(t, db, database.APIKey{
UserID: bannedUser.ID,
ExpiresAt: time.Now().Add(time.Hour),
})
// WHEN: the user loads the page.
r = httptest.NewRequest("GET", "/", nil)
r.Header.Set(codersdk.SessionTokenHeader, bannedToken)
rw = httptest.NewRecorder()
handler.ServeHTTP(rw, r)
require.Equal(t, http.StatusOK, rw.Code)
// THEN: createWorkspace = false because the ban's negative permission
// overrides the create permission granted by org membership.
var bannedPerms codersdk.AuthorizationResponse
err = json.Unmarshal([]byte(html.UnescapeString(rw.Body.String())), &bannedPerms)
require.NoError(t, err)
assert.False(t, bannedPerms["createWorkspace"], "org member with a workspace-creation ban should have createWorkspace = false")
}
func TestInjectionFailureProducesCleanHTML(t *testing.T) {
t.Parallel()
db, _ := dbtestutil.NewDB(t)
// Create an expired user with a refresh token, but provide no OAuth2
// configuration so that refresh is impossible, this should result in
// an error when httpmw.ExtractAPIKey is called.
user := dbgen.User(t, db, database.User{})
_, token := dbgen.APIKey(t, db, database.APIKey{
UserID: user.ID,
LastUsed: dbtime.Now().Add(-time.Hour),
ExpiresAt: dbtime.Now().Add(-time.Second),
LoginType: database.LoginTypeGithub,
})
_ = dbgen.UserLink(t, db, database.UserLink{
UserID: user.ID,
LoginType: database.LoginTypeGithub,
OAuthRefreshToken: "hello",
OAuthExpiry: dbtime.Now().Add(-time.Second),
})
siteFS := fstest.MapFS{
"index.html": &fstest.MapFile{
Data: []byte("<html>{{ .User }}</html>"),
},
}
handler, err := site.New(&site.Options{
Telemetry: telemetry.NewNoop(),
Database: db,
SiteFS: siteFS,
// No OAuth2 configs, refresh will fail.
OAuth2Configs: &httpmw.OAuth2Configs{
Github: nil,
OIDC: nil,
},
})
require.NoError(t, err)
r := httptest.NewRequest("GET", "/", nil)
r.Header.Set(codersdk.SessionTokenHeader, token)
rw := httptest.NewRecorder()
handler.ServeHTTP(rw, r)
// Ensure we get a clean HTML response with no user data or errors
// from httpmw.ExtractAPIKey.
assert.Equal(t, http.StatusOK, rw.Code)
body := rw.Body.String()
assert.Equal(t, "<html></html>", body)
}
func TestOrganizationsMetadata(t *testing.T) {
t.Parallel()
// GIVEN: a site handler backed by an authz-wrapped database,
// matching production wiring, and a template that renders only
// the organizations metadata.
siteFS := fstest.MapFS{
"index.html": &fstest.MapFile{
Data: []byte("{{ .Organizations }}"),
},
}
rawDB, _ := dbtestutil.NewDB(t)
db := dbauthz.New(
rawDB,
rbac.NewStrictCachingAuthorizer(prometheus.NewRegistry()),
testutil.Logger(t),
coderdtest.AccessControlStorePointer(),
)
handler, err := site.New(&site.Options{
Telemetry: telemetry.NewNoop(),
Database: db,
SiteFS: siteFS,
})
require.NoError(t, err)
ctx := testutil.Context(t, testutil.WaitShort)
// GIVEN: an org with members, another org, and a soft-deleted org.
memberOrg := dbgen.Organization(t, rawDB, database.Organization{})
otherOrg := dbgen.Organization(t, rawDB, database.Organization{})
deletedOrg := dbgen.Organization(t, rawDB, database.Organization{})
err = rawDB.UpdateOrganizationDeletedByID(ctx, database.UpdateOrganizationDeletedByIDParams{
ID: deletedOrg.ID,
UpdatedAt: dbtime.Now(),
})
require.NoError(t, err)
fetchOrgIDs := func(t *testing.T, token string) []uuid.UUID {
t.Helper()
r := httptest.NewRequest("GET", "/", nil)
r.Header.Set(codersdk.SessionTokenHeader, token)
rw := httptest.NewRecorder()
handler.ServeHTTP(rw, r)
require.Equal(t, http.StatusOK, rw.Code)
var orgs []codersdk.Organization
err := json.Unmarshal([]byte(html.UnescapeString(rw.Body.String())), &orgs)
require.NoError(t, err)
ids := make([]uuid.UUID, 0, len(orgs))
for _, org := range orgs {
ids = append(ids, org.ID)
}
return ids
}
// WHEN: an owner who is a member of only one org loads the page.
owner := dbgen.User(t, rawDB, database.User{
RBACRoles: []string{codersdk.RoleOwner},
})
dbgen.OrganizationMember(t, rawDB, database.OrganizationMember{
OrganizationID: memberOrg.ID,
UserID: owner.ID,
})
_, ownerToken := dbgen.APIKey(t, rawDB, database.APIKey{
UserID: owner.ID,
ExpiresAt: time.Now().Add(time.Hour),
})
// THEN: the metadata includes every non-deleted org, not just the
// orgs the owner is a member of.
ownerOrgIDs := fetchOrgIDs(t, ownerToken)
assert.Contains(t, ownerOrgIDs, memberOrg.ID)
assert.Contains(t, ownerOrgIDs, otherOrg.ID)
assert.NotContains(t, ownerOrgIDs, deletedOrg.ID)
// WHEN: a regular member of a single org loads the page.
member := dbgen.User(t, rawDB, database.User{})
dbgen.OrganizationMember(t, rawDB, database.OrganizationMember{
OrganizationID: memberOrg.ID,
UserID: member.ID,
})
_, memberToken := dbgen.APIKey(t, rawDB, database.APIKey{
UserID: member.ID,
ExpiresAt: time.Now().Add(time.Hour),
})
// THEN: the metadata only includes orgs the member can read.
memberOrgIDs := fetchOrgIDs(t, memberToken)
assert.Contains(t, memberOrgIDs, memberOrg.ID)
assert.NotContains(t, memberOrgIDs, otherOrg.ID)
assert.NotContains(t, memberOrgIDs, deletedOrg.ID)
}
func TestUserSecretFilePathEnabledMetadata(t *testing.T) {
t.Parallel()
// The dashboard reads this as a positive capability, so the
// metadata is the negation of the deployment's disable option.
for _, tc := range []struct {
name string
enabled bool
expected string
}{
{name: "Enabled", enabled: true, expected: "true"},
{name: "Disabled", enabled: false, expected: "false"},
} {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
// GIVEN: a site handler configured with the capability and a
// template that renders only that metadata value.
siteFS := fstest.MapFS{
"index.html": &fstest.MapFile{
Data: []byte("{{ .UserSecretFilePathEnabled }}"),
},
}
db, _ := dbtestutil.NewDB(t)
handler, err := site.New(&site.Options{
Telemetry: telemetry.NewNoop(),
Database: db,
SiteFS: siteFS,
UserSecretFilePathEnabled: tc.enabled,
})
require.NoError(t, err)
user := dbgen.User(t, db, database.User{})
_, token := dbgen.APIKey(t, db, database.APIKey{
UserID: user.ID,
ExpiresAt: time.Now().Add(time.Hour),
})
// WHEN: an authenticated user loads the page.
r := httptest.NewRequest("GET", "/", nil)
r.Header.Set(codersdk.SessionTokenHeader, token)
rw := httptest.NewRecorder()
handler.ServeHTTP(rw, r)
// THEN: the metadata renders as a JSON boolean.
require.Equal(t, http.StatusOK, rw.Code)
require.Equal(t, tc.expected, strings.TrimSpace(rw.Body.String()))
})
}
}
func TestExperimentsMetadata(t *testing.T) {
t.Parallel()
// GIVEN: a site handler whose experiments come from stored rules, and
// a template that renders only the experiments metadata.
siteFS := fstest.MapFS{
"index.html": &fstest.MapFile{
Data: []byte("{{ .Experiments }}"),
},
}
db, _ := dbtestutil.NewDB(t)
evaluator, err := experiments.New(testutil.Logger(t), experiments.NewDBStore(db), codersdk.Experiments{"foo"})
require.NoError(t, err)
handler, err := site.New(&site.Options{
Telemetry: telemetry.NewNoop(),
Database: db,
SiteFS: siteFS,
ExperimentEvaluator: evaluator,
})
require.NoError(t, err)
ctx := testutil.Context(t, testutil.WaitShort)
eligible := dbgen.User(t, db, database.User{})
other := dbgen.User(t, db, database.User{})
_, _, changed, err := experiments.WriteRule(ctx, db, eligible.ID, codersdk.ExperimentExample, experiments.Rule{
Mode: experiments.ModeCondition,
Condition: fmt.Sprintf("user.username == %q", eligible.Username),
}, 0)
require.NoError(t, err)
require.True(t, changed)
fetchExperiments := func(t *testing.T, userID uuid.UUID) codersdk.Experiments {
t.Helper()
_, token := dbgen.APIKey(t, db, database.APIKey{
UserID: userID,
ExpiresAt: time.Now().Add(time.Hour),
})
r := httptest.NewRequest("GET", "/", nil)
r.Header.Set(codersdk.SessionTokenHeader, token)
rw := httptest.NewRecorder()
handler.ServeHTTP(rw, r)
require.Equal(t, http.StatusOK, rw.Code)
var got codersdk.Experiments
require.NoError(t, json.Unmarshal([]byte(html.UnescapeString(rw.Body.String())), &got))
return got
}
// THEN: the metadata reflects the signed-in user's rules.
require.Equal(t, codersdk.Experiments{"foo", codersdk.ExperimentExample}, fetchExperiments(t, eligible.ID))
require.Equal(t, codersdk.Experiments{"foo"}, fetchExperiments(t, other.ID))
}
func TestCaching(t *testing.T) {
t.Parallel()
// Create a test server
rootFS := fstest.MapFS{
"bundle.js": &fstest.MapFile{},
"image.png": &fstest.MapFile{},
"static/image.png": &fstest.MapFile{},
"favicon.ico": &fstest.MapFile{
Data: []byte("folderFile"),
},
"service-worker.js": &fstest.MapFile{},
"index.html": &fstest.MapFile{
Data: []byte("folderFile"),
},
"terminal.html": &fstest.MapFile{
Data: []byte("folderFile"),
},
}
db, _ := dbtestutil.NewDB(t)
s, err := site.New(&site.Options{
Telemetry: telemetry.NewNoop(),
SiteFS: rootFS,
Database: db,
})
require.NoError(t, err)
srv := httptest.NewServer(s)
defer srv.Close()
// Create a context
ctx, cancelFunc := context.WithTimeout(context.Background(), testutil.WaitShort)
defer cancelFunc()
const immutable = "public, max-age=31536000, immutable"
testCases := []struct {
path string
wantCache string
}{
{"/bundle.js", immutable},
{"/image.png", immutable},
{"/static/image.png", immutable},
{"/favicon.ico", immutable},
{"/service-worker.js", ""},
// Rendered HTML embeds per-user state and must not be stored.
{"/", "no-store"},
{"/index.html", "no-store"},
{"/double/nested/terminal.html", "no-store"},
}
for _, testCase := range testCases {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, srv.URL+testCase.path, nil)
require.NoError(t, err, "create request")
res, err := srv.Client().Do(req)
require.NoError(t, err, "get index")
require.Equalf(t, testCase.wantCache, res.Header.Get("Cache-Control"), "Cache-Control of %s", testCase.path)
require.NoError(t, res.Body.Close(), "closing response")
}
}
func TestServingFiles(t *testing.T) {
t.Parallel()
// Create a test server
rootFS := fstest.MapFS{
"index.html": &fstest.MapFile{
Data: []byte("index-bytes"),
},
"favicon.ico": &fstest.MapFile{
Data: []byte("favicon-bytes"),
},
"dashboard.js": &fstest.MapFile{
Data: []byte("dashboard-js-bytes"),
},
"dashboard.css": &fstest.MapFile{
Data: []byte("dashboard-css-bytes"),
},
"install.sh": &fstest.MapFile{
Data: []byte("install-sh-bytes"),
},
}
db, _ := dbtestutil.NewDB(t)
handler, err := site.New(&site.Options{
Telemetry: telemetry.NewNoop(),
SiteFS: rootFS,
Database: db,
})
require.NoError(t, err)
srv := httptest.NewServer(handler)
defer srv.Close()
client := srv.Client()
// Create a context
ctx, cancelFunc := context.WithTimeout(context.Background(), testutil.WaitShort)
defer cancelFunc()
testCases := []struct {
path string
expected string
}{
// Index cases
{"/", "index-bytes"},
{"/index.html", "index-bytes"},
{"/nested", "index-bytes"},
{"/nested/", "index-bytes"},
{"/nested/index.html", "index-bytes"},
// These are nested paths that should lead back to index. We don't
// allow nested JS or CSS files.
{"/double/nested", "index-bytes"},
{"/double/nested/", "index-bytes"},
{"/double/nested/index.html", "index-bytes"},
{"/nested/dashboard.js", "index-bytes"},
{"/nested/dashboard.css", "index-bytes"},
{"/double/nested/dashboard.js", "index-bytes"},
{"/double/nested/dashboard.css", "index-bytes"},
// Favicon cases
// The favicon is always root-referenced in index.html:
{"/favicon.ico", "favicon-bytes"},
// JS, CSS cases
{"/dashboard.js", "dashboard-js-bytes"},
{"/dashboard.css", "dashboard-css-bytes"},
// Install script
{"/install.sh", "install-sh-bytes"},
}
for _, testCase := range testCases {
path := srv.URL + testCase.path
req, err := http.NewRequestWithContext(ctx, "GET", path, nil)
require.NoError(t, err)
resp, err := client.Do(req)
require.NoError(t, err, "get file")
data, _ := io.ReadAll(resp.Body)
require.Equal(t, string(data), testCase.expected, "Verify file: "+testCase.path)
err = resp.Body.Close()
require.NoError(t, err)
}
}
func TestShouldCacheFile(t *testing.T) {
t.Parallel()
testCases := []struct {
reqFile string
expected bool
}{
{"123456789.js", true},
{"apps/app/code/terminal.css", true},
{"image.png", true},
{"static/image.png", true},
{"static/images/section-a/image.jpeg", true},
{"service-worker.js", false},
{"dashboard.html", false},
{"apps/app/code/terminal.html", false},
}
for _, testCase := range testCases {
got := site.ShouldCacheFile(testCase.reqFile)
require.Equal(t, testCase.expected, got, fmt.Sprintf("Expected ShouldCacheFile(%s) to be %t", testCase.reqFile, testCase.expected))
}
}
const (
binCoderSha1 = "bin/coder.sha1"
binCoderTarZstd = "bin/coder.tar.zst"
)
var sampleBinSHAs = map[string]string{
"coder-linux-amd64": "55641d5d56bbb8ccf5850fe923bd971b86364604",
}
func sampleBinFS() fstest.MapFS {
sha1File := bytes.NewBuffer(nil)
for name, sha := range sampleBinSHAs {
_, _ = fmt.Fprintf(sha1File, "%s *%s\n", sha, name)
}
return fstest.MapFS{
binCoderSha1: &fstest.MapFile{
Data: sha1File.Bytes(),
},
binCoderTarZstd: &fstest.MapFile{
// echo -n compressed >coder-linux-amd64
// shasum -b -a 1 coder-linux-amd64 | tee coder.sha1
// tar cf coder.tar coder.sha1 coder-linux-amd64
// zstd --long --ultra -22 coder.tar
Data: []byte{
0x28, 0xb5, 0x2f, 0xfd, 0x64, 0x00, 0x27, 0xb5, 0x04, 0x00, 0x12, 0x08,
0x1a, 0x1a, 0x90, 0xa7, 0x0e, 0x00, 0x0c, 0x19, 0x7c, 0xfb, 0xa0, 0xa1,
0x5d, 0x21, 0xee, 0xae, 0xa8, 0x35, 0x65, 0x26, 0x57, 0x6e, 0x9a, 0xee,
0xaf, 0x77, 0x94, 0x01, 0xf8, 0xec, 0x3d, 0x86, 0x1c, 0xdc, 0xb1, 0x76,
0x8d, 0x31, 0x8a, 0x00, 0xf6, 0x77, 0xa9, 0x48, 0x24, 0x06, 0x42, 0xa1,
0x08, 0x14, 0x4e, 0x67, 0x5f, 0x47, 0x4a, 0x8f, 0xf1, 0x6a, 0x8d, 0xc1,
0x5a, 0x36, 0xea, 0xb6, 0x16, 0x52, 0x4a, 0x79, 0x7f, 0xbf, 0xb2, 0x77,
0x63, 0x4b, 0x0e, 0x4b, 0x41, 0x12, 0xe2, 0x25, 0x98, 0x05, 0x73, 0x53,
0x35, 0x71, 0xf5, 0x68, 0x37, 0xb7, 0x61, 0x45, 0x3e, 0xd9, 0x47, 0x99,
0x3d, 0x51, 0xd3, 0xe0, 0x09, 0x10, 0xf6, 0xc7, 0x0a, 0x10, 0x20, 0x50,
0x2b, 0x2e, 0x6d, 0x03, 0xf2, 0x21, 0xef, 0xc7, 0xa8, 0xc4, 0x3b, 0x8c,
0x03, 0x64, 0x1a, 0xd9, 0x9d, 0x01, 0x60, 0xac, 0x94, 0x5a, 0x08, 0x05,
0x4d, 0xb2, 0xd1, 0x0a, 0x99, 0x14, 0x48, 0xe3, 0xd9, 0x01, 0x99, 0x1d,
0xe0, 0xda, 0xd4, 0xbd, 0xd4, 0xc6, 0x51, 0x0d,
},
},
}
}
func TestServingBin(t *testing.T) {
t.Parallel()
// Create a misc rootfs for realistic test.
rootFS := fstest.MapFS{
"index.html": &fstest.MapFile{
Data: []byte("index-bytes"),
},
"favicon.ico": &fstest.MapFile{
Data: []byte("favicon-bytes"),
},
"dashboard.js": &fstest.MapFile{
Data: []byte("dashboard-js-bytes"),
},
"dashboard.css": &fstest.MapFile{
Data: []byte("dashboard-css-bytes"),
},
}
sampleBinFSCorrupted := sampleBinFS()
copy(sampleBinFSCorrupted[binCoderTarZstd].Data[10:], bytes.Repeat([]byte{0}, 10)) // Zero portion of archive.
sampleBinFSMissingSha256 := sampleBinFS()
delete(sampleBinFSMissingSha256, binCoderSha1)
type req struct {
url string
ifNoneMatch string
wantStatus int
wantBody []byte
wantOriginalSize int
wantEtag string
compression bool
}
tests := []struct {
name string
fs fstest.MapFS
reqs []req
wantErr bool
}{
{
name: "Extract and serve bin",
fs: sampleBinFS(),
reqs: []req{
{
url: "/bin/coder-linux-amd64",
wantStatus: http.StatusOK,
wantBody: []byte("compressed"),
wantOriginalSize: 10,
wantEtag: fmt.Sprintf("%q", sampleBinSHAs["coder-linux-amd64"]),
},
// Test ETag support.
{
url: "/bin/coder-linux-amd64",
ifNoneMatch: fmt.Sprintf("%q", sampleBinSHAs["coder-linux-amd64"]),
wantStatus: http.StatusNotModified,
wantOriginalSize: 10,
wantEtag: fmt.Sprintf("%q", sampleBinSHAs["coder-linux-amd64"]),
},
// Test compression support with X-Original-Content-Length
// header.
{
url: "/bin/coder-linux-amd64",
wantStatus: http.StatusOK,
wantOriginalSize: 10,
compression: true,
},
{url: "/bin/GITKEEP", wantStatus: http.StatusNotFound},
},
},
{
name: "Extract and serve bin fails due to missing sha256",
fs: sampleBinFSMissingSha256,
wantErr: true,
},
{
name: "Error on invalid archive",
fs: sampleBinFSCorrupted,
wantErr: true,
},
{
name: "Error on malformed sha256 file",
fs: fstest.MapFS{
binCoderSha1: &fstest.MapFile{Data: []byte("byebye")},
binCoderTarZstd: sampleBinFS()[binCoderTarZstd],
},
wantErr: true,
},
{
name: "Error on empty archive",
fs: fstest.MapFS{
binCoderSha1: &fstest.MapFile{Data: []byte{}},
binCoderTarZstd: &fstest.MapFile{Data: []byte{}},
},
wantErr: true,
},
{
name: "Serve local fs",
fs: fstest.MapFS{
// Only GITKEEP file on embedded fs, won't be served.
"bin/GITKEEP": &fstest.MapFile{},
},
reqs: []req{
{url: "/bin/coder-linux-amd64", wantStatus: http.StatusNotFound},
{url: "/bin/GITKEEP", wantStatus: http.StatusNotFound},
},
},
{
name: "Serve local fs when embed fs empty",
fs: fstest.MapFS{},
reqs: []req{
{url: "/bin/coder-linux-amd64", wantStatus: http.StatusNotFound},
{url: "/bin/GITKEEP", wantStatus: http.StatusNotFound},
},
},
{
name: "Serve embed fs",
fs: fstest.MapFS{
"bin/GITKEEP": &fstest.MapFile{
Data: []byte(""),
},
"bin/coder-linux-amd64": &fstest.MapFile{
Data: []byte("embed"),
},
},
reqs: []req{
// We support both hyphens and underscores for compatibility.
{
url: "/bin/coder-linux-amd64",
wantStatus: http.StatusOK,
wantBody: []byte("embed"),
wantOriginalSize: 5,
},
{
url: "/bin/coder_linux_amd64",
wantStatus: http.StatusOK,
wantBody: []byte("embed"),
wantOriginalSize: 5,
},
{
url: "/bin/GITKEEP",
wantStatus: http.StatusOK,
wantBody: []byte(""),
wantOriginalSize: 0,
},
},
},
}
//nolint // Parallel test detection issue.
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
dest := t.TempDir()
testFS := maps.Clone(rootFS)
maps.Copy(testFS, tt.fs)
handler, err := site.New(&site.Options{
Telemetry: telemetry.NewNoop(),
SiteFS: testFS,
CacheDir: dest,
})
if !tt.wantErr && err != nil {
require.NoError(t, err, "extract or read failed")
} else if tt.wantErr {
require.Error(t, err, "extraction or read did not fail")
}
compressor := middleware.NewCompressor(1, "text/*", "application/*")
srv := httptest.NewServer(compressor.Handler(handler))
defer srv.Close()
client := srv.Client()
// Create a context
ctx, cancelFunc := context.WithTimeout(context.Background(), testutil.WaitShort)
defer cancelFunc()
for _, tr := range tt.reqs {
t.Run(strings.TrimPrefix(tr.url, "/"), func(t *testing.T) {
req, err := http.NewRequestWithContext(ctx, "GET", srv.URL+tr.url, nil)
require.NoError(t, err, "http request failed")
if tr.ifNoneMatch != "" {
req.Header.Set("If-None-Match", tr.ifNoneMatch)
}
if tr.compression {
req.Header.Set("Accept-Encoding", "gzip")
}
resp, err := client.Do(req)
require.NoError(t, err, "http do failed")
defer resp.Body.Close()
gotStatus := resp.StatusCode
gotBody, _ := io.ReadAll(resp.Body)
if tr.wantStatus > 0 {
assert.Equal(t, tr.wantStatus, gotStatus, "status did not match")
}
if tr.wantBody != nil {
assert.Equal(t, string(tr.wantBody), string(gotBody), "body did not match")
}
if tr.wantStatus == http.StatusNoContent || tr.wantStatus == http.StatusNotModified {
assert.Empty(t, gotBody, "body is not empty")
}
if tr.compression {
assert.Equal(t, "gzip", resp.Header.Get("Content-Encoding"), "content encoding is not gzip")
} else {
assert.Empty(t, resp.Header.Get("Content-Encoding"), "content encoding is not empty")
}
if tr.wantEtag != "" {
assert.Equal(t, []string{tr.wantEtag}, resp.Header.Values("ETag"), "etag header values did not match")
assert.Equal(t, tr.wantEtag, resp.Header.Get("ETag"), "etag did not match")
}
if tr.wantOriginalSize > 0 {
// This is a custom header that we set to help the
// client know the size of the decompressed data. See
// the comment in site.go.
headerValues := resp.Header.Values("X-Original-Content-Length")
assert.Len(t, headerValues, 1, "X-Original-Content-Length should have exactly one value")
headerStr := resp.Header.Get("X-Original-Content-Length")
assert.NotEmpty(t, headerStr, "X-Original-Content-Length header is empty")
originalSize, err := strconv.Atoi(headerStr)
if assert.NoErrorf(t, err, "could not parse X-Original-Content-Length header %q", headerStr) {
assert.EqualValues(t, tr.wantOriginalSize, originalSize, "X-Original-Content-Length did not match")
}
}
})
}
})
}
}
func TestExtractOrReadBinFS(t *testing.T) {
t.Parallel()
t.Run("DoubleExtractDoesNotModifyFiles", func(t *testing.T) {
t.Parallel()
siteFS := sampleBinFS()
dest := t.TempDir()
_, _, err := site.ExtractOrReadBinFS(dest, siteFS)
require.NoError(t, err)
checkModtime := func() map[string]time.Time {
m := make(map[string]time.Time)
err = filepath.WalkDir(dest, func(path string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if d.IsDir() {
return nil // Only check the files.
}
stat, err := d.Info()
if err != nil {
return err
}
m[path] = stat.ModTime()
return nil
})
require.NoError(t, err)
return m
}
firstModtimes := checkModtime()
_, _, err = site.ExtractOrReadBinFS(dest, siteFS)
require.NoError(t, err)
secondModtimes := checkModtime()
assert.Equal(t, firstModtimes, secondModtimes, "second extract should not modify files")
})
t.Run("SHA256MismatchCausesReExtract", func(t *testing.T) {
t.Parallel()
siteFS := sampleBinFS()
dest := t.TempDir()
_, _, err := site.ExtractOrReadBinFS(dest, siteFS)
require.NoError(t, err)
bin := filepath.Join(dest, "bin", "coder-linux-amd64")
f, err := os.OpenFile(bin, os.O_WRONLY, 0o600)
require.NoError(t, err)
dontWant := []byte("hello")
_, err = f.WriteAt(dontWant, 0) // Overwrite the start of file.
assert.NoError(t, err) // Assert to allow f.Close.
err = f.Close()
require.NoError(t, err)
_, _, err = site.ExtractOrReadBinFS(dest, siteFS)
require.NoError(t, err)
f, err = os.Open(bin)
require.NoError(t, err)
defer f.Close()
got := make([]byte, 5) // hello
_, err = f.Read(got)
require.NoError(t, err)
assert.NotEqual(t, dontWant, got, "file should be overwritten on hash mismatch")
})
t.Run("ParsesHashes", func(t *testing.T) {
t.Parallel()
siteFS := sampleBinFS()
dest := t.TempDir()
_, hashes, err := site.ExtractOrReadBinFS(dest, siteFS)
require.NoError(t, err)
require.Equal(t, sampleBinSHAs, hashes, "hashes did not match")
})
}
func TestRenderStaticErrorPage(t *testing.T) {
t.Parallel()
d := site.ErrorPageData{
Status: http.StatusBadGateway,
Title: "Bad Gateway 1234",
Description: "shout out colin",
Actions: []site.Action{
{
Text: "Retry",
},
{
URL: "https://example.com",
Text: "Back to site",
},
},
}
rw := httptest.NewRecorder()
r := httptest.NewRequest("GET", "/", nil)
site.RenderStaticErrorPage(rw, r, d)
resp := rw.Result()
defer resp.Body.Close()
require.Equal(t, d.Status, resp.StatusCode)
require.Contains(t, resp.Header.Get("Content-Type"), "text/html")
body, err := io.ReadAll(resp.Body)
require.NoError(t, err)
bodyStr := string(body)
require.Contains(t, bodyStr, strconv.Itoa(d.Status))
require.Contains(t, bodyStr, d.Title)
require.Contains(t, bodyStr, d.Description)
require.Contains(t, bodyStr, "Retry")
require.Contains(t, bodyStr, "https://example.com")
}
func TestRenderStaticErrorPageNoStatus(t *testing.T) {
t.Parallel()
d := site.ErrorPageData{
HideStatus: true,
Status: http.StatusBadGateway,
Title: "Bad Gateway 1234",
Description: "shout out colin",
Actions: []site.Action{
{
Text: "Retry",
},
{
URL: "https://example.com",
Text: "Back to site",
},
},
}
rw := httptest.NewRecorder()
r := httptest.NewRequest("GET", "/", nil)
site.RenderStaticErrorPage(rw, r, d)
resp := rw.Result()
defer resp.Body.Close()
require.Equal(t, d.Status, resp.StatusCode)
require.Contains(t, resp.Header.Get("Content-Type"), "text/html")
body, err := io.ReadAll(resp.Body)
require.NoError(t, err)
bodyStr := string(body)
require.NotContains(t, bodyStr, strconv.Itoa(d.Status))
require.Contains(t, bodyStr, d.Title)
require.Contains(t, bodyStr, d.Description)
require.Contains(t, bodyStr, "Retry")
require.Contains(t, bodyStr, "https://example.com")
}
func TestJustFilesSystem(t *testing.T) {
t.Parallel()
tfs := fstest.MapFS{
"dir/foo.txt": &fstest.MapFile{
Data: []byte("hello world"),
},
"dir/bar.txt": &fstest.MapFile{
Data: []byte("hello world"),
},
}
mux := chi.NewRouter()
mux.Mount("/onlyfiles/", http.StripPrefix("/onlyfiles", http.FileServer(http.FS(site.OnlyFiles(tfs)))))
mux.Mount("/all/", http.StripPrefix("/all", http.FileServer(http.FS(tfs))))
// The /all/ endpoint should serve the directory listing.
resp := httptest.NewRecorder()
mux.ServeHTTP(resp, httptest.NewRequest("GET", "/all/dir/", nil))
require.Equal(t, http.StatusOK, resp.Code, "all serves the directory")
resp = httptest.NewRecorder()
mux.ServeHTTP(resp, httptest.NewRequest("GET", "/onlyfiles/dir/", nil))
require.Equal(t, http.StatusNotFound, resp.Code, "onlyfiles does not serve the directory")
}