> 🤖 This PR was written by Coder Agents on behalf of Jake Howell.
Closes
[DEVEX-381](https://linear.app/codercom/issue/DEVEX-381/flake-test-tasksendwaitsforworkingappstate).
Follow-up to #25648 and #25858, which addressed a different symptom of
the same test.
## Symptom
```
task_send_test.go:348: context expired while waiting for trap: context deadline exceeded
--- FAIL: Test_TaskSend/WaitsForWorkingAppState (26.02s)
```
Windows-only, on `test-go-pg (windows-2022)`. Reported four times since
#25648 landed (2026-06-02, 2026-06-10, 2026-07-01).
## Root cause
The test:
1. `setupCLITaskTest` inserts `workspace_app_status(state=idle)` at the
end of setup.
2. `WaitsForWorkingAppState` then inserts
`workspace_app_status(state=working)` before starting the CLI.
3. Both are persisted via `dbtime.Now()`, which rounds to microseconds.
Windows `time.Now()` resolution is coarser than that (often ~1 ms or
worse), so back-to-back calls frequently round to the same microsecond.
4. `GetLatestWorkspaceAppStatusesByWorkspaceIDs` has no tiebreaker:
```sql
ORDER BY workspace_id, created_at DESC
```
Its sibling `GetLatestWorkspaceAppStatusByAppID` already uses `ORDER BY
created_at DESC, id DESC` for exactly this reason. When the two rows
collide, Postgres picks either.
5. On the failing runs, the query returned the `idle` row.
`waitForTaskIdle` saw idle on the first poll, returned nil, `TaskSend`
proceeded, and the CLI completed successfully in ~5 s.
6. But the test was blocked at `resetTrap.MustWait(ctx)` waiting for a
**second** `ticker.Reset` that never happened. `WaitLong = 25s` elapsed,
line 348 failed.
CI log confirms the sequence: only one `Ticker.Reset(5s)` is caught,
then `Ticker.Stop([]) call, matched 0 traps` (from `defer
ticker.Stop()`), then the trap wait times out.
This is the same class of flake Spike documented in #15923 and #21332
("Windows in particular doesn't have high-resolution timers"), just
hidden behind a SQL `ORDER BY`.
## Fix
Two changes:
1. **`coderd/database/queries/workspaceapps.sql`**: add an `id DESC`
tiebreaker to `GetLatestWorkspaceAppStatusesByWorkspaceIDs`, matching
`GetLatestWorkspaceAppStatusByAppID`. Makes the query deterministic when
`created_at` collides.
2. **`cli/task_test.go` / `cli/task_send_test.go`**: add a
`withoutInitialAppStatus()` option to `setupCLITaskTest` and use it from
`WaitsForWorkingAppState`. The test now inserts a single `working` row,
so the collision cannot happen in the first place. Belt-and-braces with
change 1.
Comments in both places reference DEVEX-381 and #21332 so the next agent
doesn't have to re-derive this.
## Verification
- `go test ./cli -run 'Test_TaskSend' -count=1`: all 12 subtests pass,
`WaitsForWorkingAppState` completes in ~5.6 s (was ~16 s previously due
to a longer poll loop).
- Stress: 20 sequential runs of `WaitsForWorkingAppState` on Linux,
race-enabled binary, all pass in ~5.5 s each.
- `go test ./coderd -run 'AppStatus|Task' -count=1` passes.
- `go vet ./coderd/database/... ./cli/...` clean.
- `make lint/emdash` clean.
- `gofmt` clean.
Not reproducible on Linux (real time between the two patches is orders
of magnitude larger than microsecond); the Windows path is fixed by
making the ordering deterministic and by not creating the collision in
the first place.
<details>
<summary>Implementation plan & decision log</summary>
### Investigation
1. Pulled the failing job log for run `28483879823/job/84428355669`.
2. Traced the mock-clock trap sequence: one `NewTicker` and exactly one
`Ticker.Reset(5s)` were caught, then `Ticker.Stop([]) call, matched 0
traps` fires (the `defer ticker.Stop()` on `waitForTaskIdle` return).
This proves `waitForTaskIdle` returned after a single poll, not that the
trap machinery hung.
3. The command exited with `<nil>` (`clitest.go:299: command "coder task
send" exited with error: <nil>`) and a `POST /send` completed in 5.4 s.
So the CLI succeeded; the test's own trap wait is what timed out.
4. The only `waitForTaskIdle` return-nil paths are `Active +
CurrentState.State in {Idle, Complete, Failed}` and `Active +
CurrentState == nil past 30s grace`. First observation of nil cannot be
past 30s. So `TaskByID` must have returned `State == Idle`.
5. Traced `TaskByID` → `taskGet` → `workspaceData` →
`GetLatestWorkspaceAppStatusesByWorkspaceIDs`. Found the missing
tiebreaker; the sibling query one line above
(`GetLatestWorkspaceAppStatusByAppID`) already had it.
6. Confirmed the two `PATCH /app-status` calls in the Windows log
happened at `00:26:13.077` and `00:26:13.093`, well within Windows timer
resolution.
7. Confirmed `dbtime.Now()` rounds to microseconds; Windows `time.Now()`
doesn't have that precision, so `Round(time.Microsecond)` on two calls
close together frequently produces equal values.
### Prior art from Spike
- #15923: loosened `HeartbeatPeriod * 9/10` to `3/4` for Windows.
- #21332: switched `assert.After` to `assert.NotBefore` because
timestamps can equal on Windows.
Both explicitly cite "Windows doesn't always have high-resolution timers
available."
### Considered alternatives
- **Only fix the test.** Works today but leaves the SQL query
non-deterministic; another test that relies on
`GetLatestWorkspaceAppStatusesByWorkspaceIDs` could hit the same
collision.
- **Only fix the SQL query.** Would give a stable answer but not
necessarily the *right* one. If both patches share a `created_at`, `id
DESC` picks whichever UUID sorted higher, still random with respect to
insertion order.
- **Make `dbtime.Now()` monotonic per process.** Cleanest at the source,
but affects every timestamp in the database and has broader implications
than a targeted flake fix.
Going with both the query fix (defense in depth, matches existing
pattern) and the test fix (eliminates the collision at the source) is
the smallest change that closes the flake and hardens the query.
### Rejected commit-message scopes
Changes touch both `cli/` and `coderd/database/`, so per AGENTS.md the
scope is omitted for the cross-cutting commit and PR title.
</details>
Cleans the last few instances of ExpectMatch that didn't use the new `(ctx, ...)` variant, then deletes the deprecated method and renames `ExpectMatchContext` to drop the `Context` suffix.
The PausedDuringWaitForReady and WaitsForWorkingAppState tests flaked
because the quartz resetTrap was released immediately after catching
ticker.Reset (line 174), allowing client.TaskByID (line 175) to race
with the subsequent DB mutation (pauseTask / PatchAppStatus).
Fix: keep the resetTrap open across both poll iterations. On the first
poll, release the trap so the goroutine sees the initial state and
continues. On the second poll, hold the goroutine frozen at
ticker.Reset while mutating state. Then release; client.TaskByID
deterministically sees the mutated state. No race because the
goroutine cannot execute client.TaskByID while trapped.
Closes CODAGT-482
PausedDuringWaitForReady used the real clock, so the 5s poll in
waitForTaskIdle could race with an in-flight stop build. The SQL
view (tasks_with_status) returns "unknown" for stop builds with
job_status != "succeeded" because the build_status CASE has no
branch for (stop, pending) or (stop, running). On macOS CI, where
the provisioner is slower, the poll fires during this transient
window and hits the TaskStatusUnknown case instead of
TaskStatusPaused, failing with "task entered unknown state" rather
than the expected "was paused".
Convert to the same quartz mock clock pattern that PR #25648
applied to WaitsForWorkingAppState: inject a mock clock via
NewWithClock, trap ticker creation and reset, then advance time
deterministically so the poll fires after the stop build completes.
Closes CODAGT-482
waitForTaskIdle used time.NewTicker(5s) which delays the first poll
by 5 seconds. Debugger tracing proved the failure mechanism: on slow
CI (Windows), the first poll at 5s sees "working" (idle patch has not
landed due to goroutine scheduling), needs poll #2 at 10s, but the
25s context expires before it fires.
Two changes:
1. Use r.clock.NewTicker (quartz) with time.Nanosecond initial
interval and Reset(5s) for immediate first poll. Tests inject a
mock clock via clitest.NewWithClock for deterministic control.
2. Rewrite WaitsForWorkingAppState test with quartz traps
(NewTicker + TickerReset) for deterministic synchronization
instead of racing goroutines. Fix PausedDuringWaitForReady
sync point.
Closes DEVEX-381
Context was created before expensive setup operations (building
workspaces, starting agents), leaving insufficient time for the actual
command execution. Split into setupCtx for setup and a fresh ctx for
the command to ensure both get the full timeout.
Fixes all our Go file imports to match the preferred spec that we've _mostly_ been using. For example:
```
import (
"context"
"time"
"github.com/prometheus/client_golang/prometheus"
"golang.org/x/xerrors"
"gopkg.in/natefinch/lumberjack.v2"
"cdr.dev/slog/v3"
"github.com/coder/coder/v2/codersdk/agentsdk"
"github.com/coder/serpent"
)
```
3 groups: standard library, 3rd partly libs, Coder libs.
This PR makes the change across the codebase. The PR in the stack above modifies our formatting to maintain this state of affairs, and is a separate PR so it's possible to review that one in detail.
## Overview
This change promotes the tasks CLI commands from `coder exp task` to
`coder task`, marking them as generally available (GA).
## Migration
Users will need to update their scripts from:
```shell
coder exp task create "my task"
```
To:
```shell
coder task create "my task"
```
---
🤖 This change was written by Claude Sonnet 4.5 Thinking using [mux](https://github.com/coder/mux) and reviewed by a human 🏄🏻♂️
Jake Howell