Files
cline/.github/workflows/ext-vscode-test-e2e.yml
Workflow config file is invalid. Please check your config file: getMatrixes: matrix include must be a list of mappings
Dominic Cooney 159951f82d ci(vscode): make the e2e job spend its time on the tests (#14607)
* ci(vscode): make the e2e job spend its time on the tests

Windows e2e runs hit the 20-minute limit after their tests had passed:
the job was saving a multi-gigabyte bun cache that only that pull request
could read. The job also repeated the type check and lint that Quality
Checks already runs, and downloaded Playwright browsers the tests never
start.

- Restore the bun cache on every run and save it only from main, right
  after the install, so a later test failure keeps it.
- Build the extension with build:production and load it as the
  extension under development. The tests already ran that copy; the VSIX
  they also installed was never used, and packaging it ran
  vscode:prepublish's checks.
- Install only Playwright's ffmpeg, which it records test videos with.
  The tests launch VS Code through Electron, not a Playwright browser.
- Let runs on main finish instead of cancelling them on the next merge.
- Raise the timeout to 30 minutes for the run on main that installs
  without a cache and saves a new one.

* ci(vscode): make the e2e workflow comments match its behavior

A run on main that is still waiting to start is replaced by the next merge; only a run that has started always finishes. The comments now say so, and name the directory VS Code loads the extension from.

* ci(vscode): run Quality Checks when the Playwright config changes

The e2e build no longer runs the type check and lint, and
playwright.config.ts was the one lintable e2e input outside Quality
Checks' path filter. A change to only that file could pass CI unlinted.

* ci(vscode): stop caching Bun's install cache in e2e

Restoring the cache and installing took about as long as an uncached
`bun install` in ext-vscode-test's jobs on the same commits: 3:48-4:31
against 2:39-5:06 on Windows, and 25s against 23s on Ubuntu. With no
cache to save, a run on main can be cancelled by the next merge again
and the job fits main's 20-minute timeout, so concurrency and the
timeout go back to main's settings.

The e2e path filter also drops apps/vscode/.vscodeignore: test:e2e no
longer packages a VSIX, so nothing in the job reads that file.
2026-09-29 21:18:37 +02:00

159 lines
6.4 KiB
YAML

name: ext-vscode-test-e2e
on:
push:
branches:
- main
pull_request:
types: [opened, reopened, synchronize, ready_for_review]
workflow_dispatch:
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
permissions:
contents: read
pull-requests: read
jobs:
detect-changes:
runs-on: ubuntu-latest
name: Detect Changes
outputs:
e2e: ${{ steps.force.outputs.run_all == 'true' || steps.filter.outputs.e2e == 'true' }}
steps:
- id: force
if: github.event_name == 'push' || github.event_name == 'workflow_dispatch'
run: echo "run_all=true" >> "$GITHUB_OUTPUT"
- uses: actions/checkout@v4
if: steps.force.outputs.run_all != 'true'
- uses: dorny/paths-filter@v3
if: steps.force.outputs.run_all != 'true'
id: filter
with:
filters: |
e2e:
- 'apps/vscode/src/**'
- 'apps/vscode/webview-ui/**'
- 'apps/vscode/proto/**'
- 'apps/vscode/tests/**'
- 'apps/vscode/scripts/**'
- 'apps/vscode/standalone/**'
- 'apps/vscode/assets/**'
- 'apps/vscode/walkthrough/**'
- 'apps/vscode/package.json'
- 'apps/vscode/webview-ui/package.json'
# Root bun lockfile is authoritative for the whole workspace (incl. apps/vscode).
- 'bun.lock'
# SDK source packages are local workspace symlinks (@cline/*), so SDK changes affect the build.
- 'sdk/packages/**'
- 'apps/vscode/buf.yaml'
- 'apps/vscode/tsconfig*.json'
- 'apps/vscode/biome.jsonc'
- 'apps/vscode/esbuild.mjs'
- 'apps/vscode/bunfig.toml'
- 'apps/vscode/.vscode-test.mjs'
- 'apps/vscode/playwright*.ts'
- '.github/workflows/ext-vscode-test-e2e.yml'
matrix_prep:
needs: detect-changes
if: needs.detect-changes.outputs.e2e == 'true'
runs-on: ubuntu-latest
outputs:
matrix: ${{ steps.set-matrix.outputs.matrix }}
steps:
- id: set-matrix
run: |
echo 'matrix=[{"runner":"ubuntu"},{"runner":"windows"},{"runner":"macos"}]' >> $GITHUB_OUTPUT
e2e:
needs: [detect-changes, matrix_prep]
if: needs.detect-changes.outputs.e2e == 'true'
strategy:
fail-fast: false
matrix:
include: ${{ fromJson(needs.matrix_prep.outputs.matrix) }}
runs-on: ${{ matrix.runner }}-latest
timeout-minutes: 20
# Nothing in this job uses OIDC, so it does not need an id-token
# permission.
permissions:
contents: read
defaults:
run:
working-directory: apps/vscode
steps:
- uses: actions/checkout@v4
- name: Setup Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: "1.4.2"
# Bun's install cache is not cached. On every runner, restoring it
# and then installing took about as long as installing without it,
# as ext-vscode-test does.
# Cache VS Code installation. The key is exact-match only (no
# restore-keys prefix fallbacks); a miss just downloads VS Code.
- name: Cache VS Code
uses: actions/cache@v4
id: vscode-cache
with:
path: apps/vscode/.vscode-test
key: vscode-${{ runner.os }}-stable-${{ hashFiles('apps/vscode/.vscode-test.mjs', 'apps/vscode/package.json') }}
# Single root install resolves the entire bun workspace at once (replaces
# the per-package `npm ci` steps for apps/vscode + webview-ui).
- name: Install workspace dependencies
working-directory: ${{ github.workspace }}
run: bun install --frozen-lockfile
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# @cline/* are local workspace symlinks to source packages; build dist/
# before building the extension for E2E.
- name: Build SDK packages
working-directory: ${{ github.workspace }}
run: bun run build:sdk
- name: Assert better-sqlite3 native binary present
# Force bash: the Windows runner defaults to pwsh, which can't parse this
# POSIX test. Git Bash ships on GitHub's windows-latest images.
shell: bash
run: |
NODE_FILE="node_modules/better-sqlite3/build/Release/better_sqlite3.node"
if [ ! -f "$NODE_FILE" ]; then
echo "ERROR: better-sqlite3 native binary missing at apps/vscode/$NODE_FILE"
echo "(bun trustedDependencies postinstall likely did not run)"
exit 1
fi
echo "Found better-sqlite3 native binary: $NODE_FILE"
- name: Install xvfb on Linux
if: matrix.runner == 'ubuntu'
run: sudo apt-get update && sudo apt-get install -y xvfb
# Type checks and lint run in ext-vscode-test's Quality Checks job;
# test:e2e builds the extension without them.
- name: Run E2E tests - Linux
if: matrix.runner == 'ubuntu'
run: xvfb-run -a bun run test:e2e
- name: Run E2E tests - Non-Linux
if: matrix.runner != 'ubuntu'
run: bun run test:e2e
# Repo-root relative: the job's `working-directory` default applies to `run`
# steps only, so an apps/vscode-relative path here silently matches nothing
# and every failing run uploads no recordings at all.
- uses: actions/upload-artifact@v4
if: ${{ failure() }}
with:
name: playwright-recordings-${{ matrix.runner }}
path: |
apps/vscode/test-results/