blinkagent[bot]andblink-so[bot]
8bb80b060e
## Problem The `update workspace, new required, mutable parameter added` e2e test has been flaking consistently ([internal#1328](https://github.com/coder/internal/issues/1328)). The error: ``` Error: Timed out 5000ms waiting for expect(locator).toHaveValue(expected) Locator: getByTestId('parameter-field-Sixth parameter').locator('input') Expected string: "99" Received string: "" ``` ## Root Cause A race between page navigation and data hydration in `verifyParameters`: 1. The page navigates with `waitUntil: "domcontentloaded"` which does not wait for API responses to settle 2. React Query may serve stale cached workspace data initially (from before the update), causing the form to render with empty/old parameter values 3. The `toHaveValue` assertion uses the default `actionTimeout` of 5000ms which isn't enough time for fresh data to arrive and the form to re-render ## Fix - Switch `verifyParameters` navigation to `waitUntil: "networkidle"` to ensure API responses (workspace data, build parameters) are settled before the form renders - Increase the `toHaveValue` timeout to 15s to handle cases where dynamic parameters hydrate slowly after initial render Fixes coder/internal#1328 --------- Co-authored-by: blink-so[bot] <211532188+blink-so[bot]@users.noreply.github.com>
e2e
The structure of the end-to-end tests is optimized for speed and reliability. Not all tests require setting up a new PostgreSQL instance or using the Terraform provisioner. Deciding when to trade time for robustness rests with the developers; the framework's role is to facilitate this process.
Take a look at prior art in tests/ for inspiration. To run a test:
cd site
# Build the frontend assets. If you are actively changing
# the site to debug an issue, add `--watch`.
pnpm build
# Alternatively, build with debug info and source maps:
NODE_ENV=development pnpm vite build --mode=development
# Install the browsers to `~/.cache/ms-playwright`.
pnpm playwright:install
# Run E2E tests. You can see the configuration of the server
# in `playwright.config.ts`. This uses `go run -tags embed ...`.
pnpm playwright:test
# Run a specific test (`-g` stands for grep. It accepts regex).
pnpm playwright:test -g '<your test here>'
Using nix
If this breaks, it is likely because the flake chromium version and playwright are no longer compatible. To fix this, update the flake to get the latest chromium version, and adjust the playwright version in the package.json.
You can see the playwright version here: https://search.nixos.org/packages?channel=unstable&show=playwright-driver&from=0&size=50&sort=relevance&type=packages&query=playwright-driver
# Optionally add '--command zsh' to choose your shell.
nix develop
cd site
pnpm install
pnpm build
pnpm playwright:test
To run the playwright debugger from VSCode, just launch VSCode from the nix environment and have the extension installed.
# Optionally add '--command zsh' to choose your shell.
nix develop
code .
Enterprise tests
Enterprise tests require a license key to run.
export CODER_E2E_LICENSE=<license key>
Debugging tests
To debug a test, it is more helpful to run it in ui mode.
pnpm playwright:test-ui