mirror of
https://github.com/coder/coder.git
synced 2026-11-01 21:13:14 +08:00
Jake Howell
cfc66b5de3
> 🤖 This PR was written by Coder Agents on behalf of Jake Howell. Stack: 1. #26575 `fix(site/e2e): close mock external-auth servers in teardown` 2. #26793 `fix(site/e2e): accept 404 from external auth reset hook` 3. #26795 `fix(site/src): refresh provider state after device-flow exchange` 4. #26798 `fix(site/e2e): reset both providers in external auth hook` ← this PR 5. #26648 `chore(site/e2e): re-enable externalAuth suite` Surfaced by stress-testing the leaf state of the stack locally with `--repeat-each=N`. The suite passes on iter 1, then deterministically fails the device test on every subsequent iteration: | iter | device | web | workspace | |---|---|---|---| | 1 | ✓ 2.8s | ✓ 2.5s | ✓ 6.2s | | 2 | **✘ 30.1s** | ✓ 2.6s | ✓ 6.2s | | 3 | **✘ 30.1s** | ✓ ~2.5s | ✓ ~6s | | … | … | … | … | `resetExternalAuthKey` in `site/e2e/hooks.ts` only deletes the link for `gitAuth.webProvider`, never `gitAuth.deviceProvider`. So: ```ts path: `/api/v2/external-auth/${gitAuth.webProvider}?coder_session_token=${sessionCookie?.value}`, ``` Iter 1 of the device test creates a link for the `device` provider against the test user. The cleanup hook between tests resets `web` but not `device`. On iter 2 the page hits `/external-auth/device`, `externalAuthProviderQuery.data.authenticated` is already `true`, the page renders the authorized state immediately, and the test's `sentPending.wait()` hangs forever because the mock token endpoint is never polled. The whole test times out at 30s. Fix: reset every provider the suite exercises. Refactored the per-link delete into a Promise-returning helper so errors actually propagate; the previous `throw` inside the response and `error` callbacks crashed the worker rather than rejecting the call. Doesn't affect CI (single run per PR, fresh `--ephemeral` coderd), but it makes the suite robust to local stress testing and to anyone re-running it in the same coderd process. Filed before the un-skip lands so the un-skipped suite is repeatable from day one. Local verification: `pnpm playwright:test e2e/tests/externalAuth.spec.ts --repeat-each=20 --workers=1` → 61/61 passing in 4m48s. Same command before this fix → 1 pass, then deterministic device-test timeouts. Refs https://linear.app/codercom/issue/DEVEX-413 <details> <summary>Why a separate PR</summary> This is the fourth distinct root cause that piled up while the suite was skipped, and it's orthogonal to the other three. #26575 fixes the listener leak, #26793 fixes the 404 hook contract drift, #26795 fixes the dropped invalidation, and this PR fixes the per-test cleanup so the suite is actually repeatable. #26648 then just flips `.skip`. </details>
e2e
The structure of the end-to-end tests is optimized for speed and reliability. Not all tests require setting up a new PostgreSQL instance or using the Terraform provisioner. Deciding when to trade time for robustness rests with the developers; the framework's role is to facilitate this process.
Take a look at prior art in tests/ for inspiration. To run a test:
cd site
# Build the frontend assets. If you are actively changing
# the site to debug an issue, add `--watch`.
pnpm build
# Alternatively, build with debug info and source maps:
NODE_ENV=development pnpm vite build --mode=development
# Install the browsers to `~/.cache/ms-playwright`.
pnpm playwright:install
# Run E2E tests. You can see the configuration of the server
# in `playwright.config.ts`. This builds and runs `site/e2e/bin/coder`.
pnpm playwright:test
# Run a specific test (`-g` stands for grep. It accepts regex).
pnpm playwright:test -g '<your test here>'
Using nix
If this breaks, it is likely because the flake chromium version and playwright are no longer compatible. To fix this, update the flake to get the latest chromium version, and adjust the playwright version in the package.json.
You can see the playwright version here: https://search.nixos.org/packages?channel=unstable&show=playwright-driver&from=0&size=50&sort=relevance&type=packages&query=playwright-driver
# Optionally add '--command zsh' to choose your shell.
nix develop
cd site
pnpm install
pnpm build
pnpm playwright:test
To run the playwright debugger from VSCode, just launch VSCode from the nix environment and have the extension installed.
# Optionally add '--command zsh' to choose your shell.
nix develop
code .
Enterprise tests
Enterprise tests require a license key to run.
export CODER_E2E_LICENSE=<license key>
Debugging tests
To debug a test, it is more helpful to run it in ui mode.
pnpm playwright:test-ui