Commit Graph
4 Commits
Author SHA1 Message Date
Ethan 5088b5fa5f ci: extend flake-go bump test-count to 35 (#25981)
Two changes to make the `flake-go` workflow produce better signal when
something hangs or flakes at low rates.

**Job timeout (20m → 25m).** The Go-level `-timeout 20m` baked into
`make test` (`Makefile:1428`) currently raced the runner's 20m
hard-kill, so a hanging test got SIGTERM'd by Actions instead of
SIGQUIT'd by Go, and we never got the goroutine dump. Bumping the
workflow job to 25m mirrors the layering already used by `test-go-pg` in
`ci.yaml:409` and gives Go's timeout the 5m head start it expects.

**Test count (25 → 35).** Catches lower-frequency flakes that 25
attempts miss too often. For a 5% per-run flake, detection probability
goes from ~72% at n=25 to ~83% at n=35; for 1–2% flakes the lift is
larger. The longest successful flake-go run to date was 11m49s at n=25,
so n=35 should peak around ~16–17m and stay well inside the new 25m
budget.
2026-06-03 00:20:42 +10:00
Thomas KosiewskiandClaude Opus 4.8 550aa6d6a2 ci: install gotestsum in flake check workflow (#25934)
The Flake Check workflow runs `make test` through the `test-go-pg`
action, which invokes `gotestsum`, but the workflow never installs it.
The mise refactor (#25727) deleted the `setup-go` action that previously
installed `gotestsum` implicitly, and added explicit `mise install ...
go:gotest.tools/gotestsum` steps to every other Go test job. The flake
check's `Install Go mise tools` step only listed `whichtests`, so the
check fails with `gotestsum: command not found` whenever it selects
changed tests to run.

Add `go:gotest.tools/gotestsum` to the flake check's install step,
matching the other `test-go-pg` jobs in `ci.yaml` and
`nightly-gauntlet.yaml`.

Refs #25727

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 12:11:00 +10:00
Thomas Kosiewski fe257666d7 ci: refactor CI to use mise for shared tool setup (#25727) 2026-06-01 15:55:19 +02:00
Ethan ca7f07142e ci: add Go test flake detector workflow (#25667)
Adds a `flake-go` workflow that hunts for ordering-dependent and racy Go
tests on pull requests. The workflow runs only on PRs (cancelling
earlier runs on new commits) and skips test execution when no Go test
files changed.

A single `flake_go` job uses
[coder/whichtests](https://github.com/coder/whichtests) with
`--coalesce` to compute the directly-modified `Test*` functions from the
PR diff and emit them as one target row. The same job then runs those
selected tests on a deliberately resource-constrained 4-vCPU runner with
4x parallelism oversubscription, `-count=25`, and `-shuffle=on` to
amplify contention and surface flakes.

Pinned at
[coder/whichtests@ec33bab](https://github.com/coder/whichtests/commit/ec33bab1ec04cd86beb7a61a069db4463dba63f5).

Reuses the `test-go-pg` composite (with its new `run-regex`,
`test-shuffle`, and `gotestsum-json-file` inputs) and the
`go-test-failure-report` composite, both introduced on the base branch
(#25670), so this workflow shares one implementation of the gotestsum +
failure-report path with the existing CI jobs.

`Makefile` adds `TEST_SHUFFLE` support and single-quotes `RUN` so
whichtests' regex survives shell parsing.

Stacked on top of #25670.

Demo @
https://github.com/coder/coder/actions/runs/26494322649/job/78018779381?pr=25667

Closes CODAGT-381
2026-05-28 12:35:37 +10:00