Main landed the same httpapi design (git-opt-in fixtures, --shard flag)
and the same permission.create readiness wait. Resolution adopts main's
implementations and keeps this branch's unique work on top: the
--shards child-process orchestrator (adapted to main's zero-based
shard indexing), scenario retry with backoff gated on state-reset,
project.update isolation, and exerciser env stripping. Removed the
superseded agentsReady/probe machinery and this branch's duplicate
round-robin filter (double-sharding silently dropped scenarios).
Validated: coverage, auth, and 4-shard effect passes all green
(315 checks, 0 failures).
--update-durations now disables batching so every file is measured
per-file (batched members' entries otherwise rot), and only passing
runs record durations (a timed-out file would write the kill deadline
and permanently skew LPT).
Timed-out work items are no longer retried: the deadline already
proves a hang, and a second 600s attempt could push a shard past the
45-minute job budget.
A flaky batch now annotates the member files that failed the first
attempt instead of the pseudo-file. Defender exclusions apply per
path instead of aborting on the first policy-locked one.
FAST_TIERS drops the single-field wrapper left from the hand-weight era.
The unsafe-marker scan reads files in chunks of 64 so a low soft fd
limit cannot fail startup. The durations comment points at
--update-durations instead of a workflow that no longer exists.
Replacing the file wiped every batched member's measured duration —
exactly the data the computed batch weights read. Fresh per-file
timings now merge over the existing entries; deleted files drop out.
Batch shard weights are now the sum of member durations (hint >
measured > typical boot-dominated file) plus one process boot, replacing
hand-tuned per-tier constants that had gone stale (fast-tier-misc
declared 30s of ~86s real work; fast-tier-tool 55s of ~250s on Windows).
Review fixes: tool/shell, tool/task, and kilocode/daemon leave the
batches (real-work-dominated or subprocess-timing files); weight() is
memoized (it runs inside sort comparators, each miss a stat syscall);
timeout messages report the deadline actually applied to the work item;
the module-scope env-write detector also matches bracket assignments;
--update-durations refreshes the measured-durations file after a full
run; entry matching is one shared helper.
Shard children now strip KILO_HTTPAPI_EXERCISE_* so a pinned database
or global root cannot make concurrent shards collide on shared state.
The scenario retry gains a second, longer backoff (1s then 3s): sharded
children run fewer scenarios each, so early scenarios can land before
agent projections warm and one second is not always enough.
Tier entries are directory prefixes; a new test file batches (or not)
by location alone. The hand-kept exclude list shrinks to the four
timing-sensitive entries a source scan cannot detect; files with
detectable process-wide markers (mock.module, AppRuntime.dispose,
global fetch spies, module-scope env writes) are demoted to per-file
runs automatically instead of failing the suite.
The worker queue sorted by raw file weight, which is 0 for batch
pseudo-files (not real paths) — batches started last, leaving one
worker running a full batch after the rest of the shard drained.
Sorting by shard weight starts the heaviest items first and shortens
the tail.
The batch-safety guard now reads member sources in parallel, and the
diagnostics-only artifact upload no longer fails a green job on a
runner-side network blip (seen as ECONNREFUSED on Blacksmith).
The new guard correctly flagged four files inside batched subdirectories
(mock.module in kilocode/presence, global fetch spies in kilocode/sessions)
that had been batch-safe only by luck of batch composition. They run
per-file now; the exclude set applies to every tier, not just the root.
Three probes proved cli/ (minus run/), tool/, and the small acp/lsp/mcp/
plugin/storage/... dirs isolation-safe in a shared process (1052 tests,
0 failures). Work items drop from 327 to 233.
Sharding now weights unbatched files by measured durations
(script/kilocode/test-durations.json, parsed from a full local run)
instead of byte size, so LPT packs shards evenly.
The batch builder now refuses to batch files with process-wide markers
(mock.module, AppRuntime.dispose, global fetch spies) so a future test
cannot silently poison a shared process.
issue-8656-stall waits on a 90s real-time stall window; under batch
CPU contention on the Windows runner it misses the window. It joins
the timing-sensitive exclusion class and runs per-file again.
194 of 211 top-level kilocode/*.test.ts files are isolation-safe in a
shared process (validated: 1612 tests, 0 failures in one superset run).
The 17 exclusions are files that mutate process-wide state: bun
mock.module (process-wide and permanent), AppRuntime.dispose() in
teardown, global fetch spies, and real subprocess/watcher timing.
Work items drop from 506 to 327; each root batch runs ~50 files for
one process spawn instead of fifty.
permission-origins.test.ts asserts the merged config carries no
global-scope keys, so it cannot share an XDG root with batch-mates
that write global config (seen on CI as a leaked '*': 'global' bash
rule). The directory returns to per-file isolation.
Extends the fast tier from 37 to 161 files (~25% of the suite) across
four independently-scheduled batches, each empirically verified to pass
in a shared process (probes: 449 + 497 + 184 tests, zero failures):
- fast-tier-core: unchanged 37-file batch
- fast-tier-kilocode: 51 files (config, permission, provider, util, ...)
- fast-tier-kilocode-sessions: 49 files (session-export, sessions, session)
- fast-tier-kilocode-tools: 24 files (tool, cloud, anaconda-desktop)
Multiple batches instead of one keep each batch short enough to
schedule like a normal heavy file; LPT spreads them across shards and
worker slots. ~157 fewer process spawns per run - the dominant unit
cost on Windows (CreateProcessW + Defender per spawn).
The 208 top-level kilocode/*.test.ts files were probed and are NOT
batch-safe (536 cross-contamination failures in a shared process) -
they stay per-file until the #12998 singleton work lands.
Full suite locally: 505/506 with all four batches green (the one
failure is the pre-existing run-process SIGINT timeout that fails solo
on unmodified main on this machine).
Part of #12986 (in-process fast tier).
The per-file process model exists to contain cross-test state, but it
taxes every file with a process boot + TypeScript compile even when the
file is pure. This adds a fast tier: 14 directories/files empirically
verified to pass together in a single 'bun test' pass (~37 files, all
of util/, config/, effect/, patch/, provider transforms, and other
boot-dominated suites; 1360-test superset probe ran clean).
- The batch enters LPT shard splitting as one pseudo-file with a
duration-scale weight, so exactly one shard executes it and balancing
is preserved.
- JUnit: the batch writes one XML containing all member suites; the
existing merge picks it up by work-item name.
- Batching disables itself under --profile (macOS curated run) and
positional pattern filters, and the batch gets a roomier process
deadline since it compiles once for ~37 files.
- I/O-heavy directories (git/, lsp/, storage/, auth/) stay per-file:
their runtime is real work that parallelizes across the worker pool,
while a batch runs members sequentially.
Validated locally: full suite 626/627 with the batch green (the one
failure is cli/run/run-process.test.ts SIGINT timing out solo on
unmodified main on this machine - pre-existing, unrelated).
Completes #12999 (with the fixture change in the previous commit).
Adds --shard <i>/<n> (round-robin scenario subset) and --shards <n>
(spawns n child processes, each with its own PID-keyed database and
global root, tagged output, combined exit code). test:httpapi runs the
effect pass with --shards 2.
Route coverage (missing/extra) is still computed against the full
scenario list in every child, so --fail-on-missing stays correct.
Local effect-mode wall clock: 4m11s -> 2m39s (6m37s before this PR).
315/315 scenarios pass across both shards.
Windows CLI unit tests ran 4-at-a-time on the 4-vCPU runner (custom
test-runner default = min(4, cpus)), oversubscribing CPU so heavy
real-server test files blew their per-test timeouts.
- Cap KILO_TEST_CONCURRENCY=2 on Windows (each file gets ~2 vCPU).
- Grow Windows shards 4 -> 6 to absorb the lower per-shard parallelism.
- Shard by observed duration instead of file size, so the two heaviest
files no longer stack in one shard.
- Raise the Windows per-file kill deadline to 600s (KILO_TEST_FILE_TIMEOUT);
the heaviest file runs ~270s, only ~30s under the old 300s default.
Scoped to the @kilocode/cli custom runner on Windows only. Linux/macOS
unchanged; the non-CLI/core suite runs via plain 'bun test' and is not
affected (separate lever).
* test(cli): cover TUI startup outside package
* fix(cli): use native preload path in TUI test
---------
Co-authored-by: Johnny Eric Amancio <johnnyeric@gmail.com>
marius-kilocode