mirror of
https://github.com/coder/coder.git
synced 2026-09-24 15:04:27 +08:00
fix: retry and cache e2e Coder release downloads to reduce test-e2e ssh flake (#27470)
closes DEVEX-651 ## Summary Fixes coder/internal#218 (`flake: e2e-test / test ssh`). Despite the title, the `ssh with client v2.8.0` / `ssh with agent v2.12.1` cases (`site/e2e/tests/outdatedCLI.spec.ts`, `outdatedAgent.spec.ts`) are not failing because of a bug in SSH. They fail during **setup**, in `downloadCoderVersion()`, which runs `install.sh` to fetch an old Coder release from GitHub. Transient GitHub errors (HTTP 403/503, surfacing as nonzero `curl` exit codes such as 22 or 1) make `install.sh` fail and take the whole ssh test down with it. This is an external-download flake, confirmed by the recurring `install.sh failed with code {22,1}` evidence in the issue thread and Ethan's note ("Networking issues again"). ## Changes 1. **Retry-with-backoff** (`site/e2e/helpers.ts`): `downloadCoderVersion()` now retries `install.sh` up to 5 times with exponential backoff and jitter (~1s, 2s, 4s, 8s). A single transient download failure no longer fails the test. `install.sh` already reuses completed binaries and resumes partial downloads (`curl -C -`), so retries are cheap. 2. **Cross-run cache** (`.github/workflows/ci.yaml`): the `test-e2e` job now persists `/tmp/coder-e2e-cache` with `actions/cache`, so most runs skip the GitHub download entirely. The key is derived from the spec files that pin the downloaded versions, so it invalidates when those versions change. Saves are restricted to `main` (`restore` runs everywhere), matching the existing cache-poisoning convention used for the Vale and golangci-lint caches. Before this change, neither retry, mirror, nor cross-run caching protected this path; the only caching was within a single run. ## Testing - `biome check e2e/helpers.ts` passes. - `tsc --noEmit` introduces no new errors. - CI `test-e2e` exercises the changed path. <details> <summary>Investigation notes</summary> - The failure always originates in `downloadCoderVersion` -> `install.sh` -> `fetch()` (`curl -#fL ... https://github.com/coder/coder/releases/download/vX.Y.Z/...`). - `curl` exit 22 = server returned an HTTP error (403 seen in logs); exit 1 = other transient failure. GitHub also returned 503s across the workflow in some occurrences. - `/tmp/coder-e2e-cache` was not persisted by any `actions/cache` step in `ci.yaml`, so every fresh job re-downloaded from GitHub and was exposed to the flake. - Retry addresses transient failures; the cache removes the dependency on GitHub for most runs. Combined, they target the root cause at two layers. </details> --- This PR was generated by Coder Agents on behalf of @aqandrew.
This commit is contained in:
@@ -984,6 +984,19 @@ jobs:
|
||||
- run: pnpm playwright:install
|
||||
working-directory: site
|
||||
|
||||
# Cache the Coder release binaries downloaded by the outdatedCLI /
|
||||
# outdatedAgent e2e tests so most runs skip the flaky GitHub release
|
||||
# download entirely. The cache key is keyed off the test files that pin
|
||||
# the downloaded versions, so it invalidates when those versions change.
|
||||
- name: Restore e2e Coder release binary cache
|
||||
id: coder-e2e-cache
|
||||
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
||||
with:
|
||||
path: /tmp/coder-e2e-cache
|
||||
key: coder-e2e-cache-${{ runner.os }}-${{ hashFiles('site/e2e/tests/outdatedCLI.spec.ts', 'site/e2e/tests/outdatedAgent.spec.ts') }}
|
||||
restore-keys: |
|
||||
coder-e2e-cache-${{ runner.os }}-
|
||||
|
||||
# Run tests that don't require a premium license without a premium license
|
||||
- run: pnpm playwright:test --forbid-only --workers 1
|
||||
if: ${{ !matrix.variant.premium }}
|
||||
@@ -1000,6 +1013,16 @@ jobs:
|
||||
CODER_E2E_REQUIRE_PREMIUM_TESTS: "1"
|
||||
working-directory: site
|
||||
|
||||
- name: Save e2e Coder release binary cache
|
||||
# Only the default branch is trusted to write the cache, so PR runs
|
||||
# cannot poison the cache that subsequent runs restore from. Skip when
|
||||
# the cache already had an exact key hit (no new content).
|
||||
if: always() && github.ref == 'refs/heads/main' && steps.coder-e2e-cache.outputs.cache-hit != 'true'
|
||||
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
||||
with:
|
||||
path: /tmp/coder-e2e-cache
|
||||
key: ${{ steps.coder-e2e-cache.outputs.cache-primary-key }}
|
||||
|
||||
- name: Upload Playwright failure artifacts
|
||||
if: failure() && github.actor != 'dependabot[bot]' && runner.os == 'Linux' && !github.event.pull_request.head.repo.fork
|
||||
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
|
||||
|
||||
Reference in New Issue
Block a user