José Braulio González Valido 5ec90c5331 test(ai-builder): Pin threadIds in the eval-results dispatcher contract (no-changelog) (#36884) ...

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

2026-08-24 13:19:28 +00:00

..

fixtures

fix(ai-builder): Keep model-provider outages out of eval builder scores (no-changelog) (#35293)

2026-08-04 10:52:11 +00:00

agent-report-rendering.test.ts

refactor(ai-builder): Decompose the eval CLI into phase modules with typed seams (no-changelog) (#34691)

2026-07-22 16:25:44 +00:00

agent-scenario-artifact.test.ts

refactor(ai-builder): Split the eval harness runner into domain modules (no-changelog) (#34834)

2026-07-27 09:01:09 +00:00

aggregator.test.ts

feat: Add the credential-setup browser eval lane (no-changelog) (#35982)

2026-08-13 15:56:24 +00:00

args.test.ts

docs(ai-builder): Add a source-linking provenance step to the eval-authoring skill (no-changelog) (#34721)

2026-07-22 22:35:13 +00:00

artifact-agent-handler.test.ts

test(ai-builder): Support evals for agent building and for config evals (no-changelog) (#33888)

2026-07-15 12:34:21 +00:00

artifact-config-eval-handler.test.ts

chore(ai-builder): Typecheck the eval harness and pin its external contracts (no-changelog) (#34673)

2026-07-22 08:18:41 +00:00

artifact-context.test.ts

test(ai-builder): Support evals for agent building and for config evals (no-changelog) (#33888)

2026-07-15 12:34:21 +00:00

artifact-workflow-handler.test.ts

chore(ai-builder): Typecheck the eval harness and pin its external contracts (no-changelog) (#34673)

2026-07-22 08:18:41 +00:00

assertion-judge.test.ts

test(ai-builder): Support evals for agent building and for config evals (no-changelog) (#33888)

2026-07-15 12:34:21 +00:00

attribution.test.ts

feat: Add the credential-setup browser eval lane (no-changelog) (#35982)

2026-08-13 15:56:24 +00:00

browser-lane-serialisation.test.ts

feat: Add the credential-setup browser eval lane (no-changelog) (#35982)

2026-08-13 15:56:24 +00:00

bucket-from-evaluation.test.ts

fix: Report an unverifiable unit as notVerified, not a silent pass (#34421)

2026-07-17 14:23:10 +00:00

build-cost-report.test.ts

feat(ai-builder): Report per-case claude build cost for MCP eval builds (no-changelog) (#34777)

2026-07-23 18:42:09 +00:00

build-expectations-verifier.test.ts

test(ai-builder): Support evals for agent building and for config evals (no-changelog) (#33888)

2026-07-15 12:34:21 +00:00

build-orchestrator.test.ts

feat: Add the credential-setup browser eval lane (no-changelog) (#35982)

2026-08-13 15:56:24 +00:00

build-workflow-conversation-seed.test.ts

fix(core): Remember skipped Instance AI credential setup for the rest of a thread (#36308)

2026-08-17 12:53:22 +00:00

build-workflow-seed-cleanup.test.ts

test(ai-builder): Let a case declare a credential already broken before the conversation started (no-changelog) (#36029)

2026-08-13 13:33:11 +00:00

capture-run-debug.test.ts

ci: Bound and instrument Instance AI eval lane containers (no-changelog) (#33903)

2026-07-14 09:28:46 +00:00

case-files-schema.test.ts

test(core): Extend Instance AI intent eval fixtures (no-changelog) (#33224)

2026-07-07 12:49:32 +00:00

case-pipeline.test.ts

fix(ai-builder): Give a case's scenario data tables per-run names (no-changelog) (#35543)

2026-08-05 12:48:00 +00:00

checklist-verifier.test.ts

feat(core): Improve Instance AI model experiments (no-changelog) (#35034)

2026-08-13 10:15:03 +00:00

ci-metadata.test.ts

refactor(ai-builder): Decompose the eval CLI into phase modules with typed seams (no-changelog) (#34691)

2026-07-22 16:25:44 +00:00

cleanup-build.test.ts

feat(ai-builder): Support seeding agents alongside workflows in eval threads (no-changelog) (#35548)

2026-08-06 12:40:14 +00:00

collect-expectations.test.ts

feat(core): Make instanceAI evals more mcp-friendly (no-changelog) (#32916)

2026-06-25 06:33:50 +00:00

comparison-compare.test.ts

fix(ai-builder): Keep model-provider outages out of eval builder scores (no-changelog) (#35293)

2026-08-04 10:52:11 +00:00

comparison-format.test.ts

chore(ai-builder): Typecheck the eval harness and pin its external contracts (no-changelog) (#34673)

2026-07-22 08:18:41 +00:00

comparison-gate.test.ts

ci(ai-builder): Gate the mcp-gate eval tier absolutely (no-changelog) (#35762)

2026-08-06 14:37:04 +00:00

comparison-statistics.test.ts

feat(ai-builder): Add per-PR eval regression detection vs LangSmith baseline (#29456)

2026-05-06 08:15:08 +00:00

conversation-seed.test.ts

fix(core): Remember skipped Instance AI credential setup for the rest of a thread (#36308)

2026-08-17 12:53:22 +00:00

conversation-text.test.ts

feat(ai-builder): Support seeding agents alongside workflows in eval threads (no-changelog) (#35548)

2026-08-06 12:40:14 +00:00

credential-setup-checks.test.ts

test(core): Add openai, gemini and slack credential-setup eval fixtures (no-changelog) (#36461)

2026-08-21 06:27:16 +00:00

credential-setup-lane.test.ts

feat: Add the credential-setup browser eval lane (no-changelog) (#35982)

2026-08-13 15:56:24 +00:00

data-discovery.test.ts

feat(ai-builder): Harden the eval harness — crash-recovery journal, parity fixes, extension points (no-changelog) (#34747)

2026-07-23 12:25:55 +00:00

data-workflows-schema.test.ts

feat(ai-builder): Push a seeded case, and hand the agent its workflow (no-changelog) (#35304)

2026-08-05 16:09:01 +00:00

data-workflows.test.ts

chore(ai-builder): Typecheck the eval harness and pin its external contracts (no-changelog) (#34673)

2026-07-22 08:18:41 +00:00

dataset-sync.test.ts

feat(ai-builder): Harden the eval harness — crash-recovery journal, parity fixes, extension points (no-changelog) (#34747)

2026-07-23 12:25:55 +00:00

effective-timeout.test.ts

refactor(ai-builder): Split the eval harness runner into domain modules (no-changelog) (#34834)

2026-07-27 09:01:09 +00:00

eval-artifact-output-dir.test.ts

fix(ai-builder): Route eval report artifacts through --output-dir (no-changelog) (#35083)

2026-07-29 11:04:42 +00:00

eval-results-dispatcher-contract.test.ts

test(ai-builder): Pin threadIds in the eval-results dispatcher contract (no-changelog) (#36884)

2026-08-24 13:19:28 +00:00

event-parser.test.ts

fix(core): Remember skipped Instance AI credential setup for the rest of a thread (#36308)

2026-08-17 12:53:22 +00:00

execution-scenario-seed-data-tables.test.ts

feat: Add typed seed data tables with rows on execution scenarios (#34420)

2026-07-17 17:30:19 +00:00

export-latest-verifier-request.test.ts

feat(core): Harden Instance AI eval attribution and diagnostics (no-changelog) (#31951)

2026-06-09 12:10:10 +00:00

external-workflow-edits.test.ts

test(core): Drive external workflow edits from the user proxy (#35736)

2026-08-12 08:42:20 +00:00

fetch-baseline.test.ts

ci: Bound and instrument Instance AI eval lane containers (no-changelog) (#33903)

2026-07-14 09:28:46 +00:00

fixture-server.test.ts

fix(core): Stop code editors mangling values typed by browser_type (no-changelog) (#36804)

2026-08-21 13:08:05 +00:00

fixtures.ts

feat(core): Extend evaluations to support node groups (no-changelog) (#35629)

2026-08-19 08:42:53 +00:00

iterations.test.ts

refactor(ai-builder): Decompose the eval CLI into phase modules with typed seams (no-changelog) (#34691)

2026-07-22 16:25:44 +00:00

lane-allocator.test.ts

refactor(ai-builder): Decompose the eval CLI into phase modules with typed seams (no-changelog) (#34691)

2026-07-22 16:25:44 +00:00

lane-users.test.ts

test(core): Support credential seeding in MCP workflow evals (no-changelog) (#35782)

2026-08-10 19:56:03 +00:00

langsmith-seed.test.ts

refactor(core): Consolidate agent workspace string replacement tools (no-changelog) (#36178)

2026-08-17 11:11:51 +00:00

langtracer-client.test.ts

feat(core): Add eval:langtracer-push to push eval cases to a suite (no-changelog) (#33781)

2026-07-08 07:52:23 +00:00

langtracer-normalize.test.ts

feat(ai-builder): Collapse the eval case's three seed keys into one seed union (no-changelog) (#35292)

2026-08-03 14:37:06 +00:00

langtracer-provider.test.ts

feat(ai-builder): Collapse the eval case's three seed keys into one seed union (no-changelog) (#35292)

2026-08-03 14:37:06 +00:00

langtracer-push.test.ts

feat(ai-builder): Support seeding agents alongside workflows in eval threads (no-changelog) (#35548)

2026-08-06 12:40:14 +00:00

langtracer-to-exported.test.ts

feat(ai-builder): Support seeding agents alongside workflows in eval threads (no-changelog) (#35548)

2026-08-06 12:40:14 +00:00

mcp-builder-timeout.test.ts

test: Parallelize MCP workflow evals across build lanes (no-changelog) (#33468)

2026-07-03 08:33:42 +00:00

mcp-builder.test.ts

test(core): Support credential seeding in MCP workflow evals (no-changelog) (#35782)

2026-08-10 19:56:03 +00:00

mcp-registry-seeder.test.ts

fix(core): Require an authenticated session for the MCP registry test seed endpoint (no-changelog) (#34880)

2026-07-27 13:20:38 +00:00

missing-workflow-error.test.ts

refactor(ai-builder): Split the eval harness runner into domain modules (no-changelog) (#34834)

2026-07-27 09:01:09 +00:00

n8n-client-data-tables.test.ts

fix(ai-builder): Give a case's scenario data tables per-run names (no-changelog) (#35543)

2026-08-05 12:48:00 +00:00

normalize-workflow.test.ts

chore: Migrate instance-ai from Jest to Vitest (#31463)

2026-06-03 06:48:27 +00:00

pair-records.test.ts

feat: Add pairwise workflow eval pipeline (#29123)

2026-05-04 13:26:27 +00:00

partial-results-abort.test.ts

refactor(ai-builder): Split the eval harness runner into domain modules (no-changelog) (#34834)

2026-07-27 09:01:09 +00:00

prebuilt-coverage.test.ts

feat(ai-builder): Source eval test cases from LangTracer (no-changelog) (#33067)

2026-06-29 10:17:49 +00:00

prebuilt-workflows.test.ts

feat(core): Allow deleting pre-built workflows after evaluating (no-changelog) (#31960)

2026-06-09 08:50:31 +00:00

redact.test.ts

feat: Add Simplified Custom Auth credential with recipe-guided setup in the AI Assistant (#35056)

2026-08-07 09:22:36 +00:00

relay-connection.test.ts

test(core): Add openai, gemini and slack credential-setup eval fixtures (no-changelog) (#36461)

2026-08-21 06:27:16 +00:00

reshape.test.ts

fix(ai-builder): Keep model-provider outages out of eval builder scores (no-changelog) (#35293)

2026-08-04 10:52:11 +00:00

row-sink.test.ts

feat(ai-builder): Harden the eval harness — crash-recovery journal, parity fixes, extension points (no-changelog) (#34747)

2026-07-23 12:25:55 +00:00

rows.test.ts

feat(ai-builder): Harden the eval harness — crash-recovery journal, parity fixes, extension points (no-changelog) (#34747)

2026-07-23 12:25:55 +00:00

run-debug-report.test.ts

feat(ai-builder): Collapse the eval case's three seed keys into one seed union (no-changelog) (#35292)

2026-08-03 14:37:06 +00:00

runner-thread-id.test.ts

refactor(ai-builder): Split the eval harness runner into domain modules (no-changelog) (#34834)

2026-07-27 09:01:09 +00:00

runner-workflow-checks.test.ts

refactor(ai-builder): Split the eval harness runner into domain modules (no-changelog) (#34834)

2026-07-27 09:01:09 +00:00

sandbox-config.test.ts

feat(core): Use n8n default sandbox for Instance AI (no-changelog) (#31335)

2026-06-04 08:31:27 +00:00

scenario-budget-stop.test.ts

fix(ai-builder): Bound eval lane requests and stop abandoned mocked executions (no-changelog) (#35228)

2026-08-06 08:23:37 +00:00

scenario-seed-tables.test.ts

fix(ai-builder): Give a case's scenario data tables per-run names (no-changelog) (#35543)

2026-08-05 12:48:00 +00:00

select-expectations.test.ts

fix(ai-builder): Keep model-provider outages out of eval builder scores (no-changelog) (#35293)

2026-08-04 10:52:11 +00:00

stub-services.test.ts

feat: Add pairwise workflow eval pipeline (#29123)

2026-05-04 13:26:27 +00:00

stub-workspace.test.ts

test(core): Add MCP registry discovery evals (no-changelog) (#36063)

2026-08-18 12:28:05 +00:00

subagent-runner.test.ts

refactor(ai-builder): Split the eval harness runner into domain modules (no-changelog) (#34834)

2026-07-27 09:01:09 +00:00

summary.test.ts

chore(ai-builder): Typecheck the eval harness and pin its external contracts (no-changelog) (#34673)

2026-07-22 08:18:41 +00:00

transcript-from-events.test.ts

fix(core): Remember skipped Instance AI credential setup for the rest of a thread (#36308)

2026-08-17 12:53:22 +00:00

transient-error.test.ts

fix(ai-builder): Bound eval lane requests and stop abandoned mocked executions (no-changelog) (#35228)

2026-08-06 08:23:37 +00:00

user-proxy.test.ts

feat(ai-builder): Let the eval harness mint Simplified Custom Auth credentials from setup-card recipes (no-changelog) (#36192)

2026-08-19 08:44:23 +00:00

verification-artifact.test.ts

refactor(ai-builder): Split the eval harness runner into domain modules (no-changelog) (#34834)

2026-07-27 09:01:09 +00:00

verifier.test.ts

feat(core): Harden Instance AI eval attribution and diagnostics (no-changelog) (#31951)

2026-06-09 12:10:10 +00:00

workflow-checks-applicability.test.ts

chore(ai-builder): Typecheck the eval harness and pin its external contracts (no-changelog) (#34673)

2026-07-22 08:18:41 +00:00

workflow-checks-errors.test.ts

fix: Improve AI Assistant post-build follow-ups (#33419)

2026-07-02 11:34:21 +00:00

workflow-checks-feedback.test.ts

test(ai-builder): Rename response-matches eval check and fix multi-turn wiring (no-changelog) (#32810)

2026-07-10 12:58:13 +00:00

workflow-context.test.ts

feat(core): Extend evaluations to support node groups (no-changelog) (#35629)

2026-08-19 08:42:53 +00:00

workflow-discovery.test.ts

ci: Run PR pipeline on Node 26 (#34032)

2026-08-14 07:54:25 +00:00

workflow-expected.test.ts

refactor(ai-builder): Split the eval harness runner into domain modules (no-changelog) (#34834)

2026-07-27 09:01:09 +00:00

workflow-report.test.ts

fix(ai-builder): Keep model-provider outages out of eval builder scores (no-changelog) (#35293)

2026-08-04 10:52:11 +00:00

workspace-tool-contract.test.ts

feat(core): Add writable workspaces to agents (#36033)

2026-08-12 12:24:33 +00:00

write-eval-results.test.ts

refactor(ai-builder): Decompose the eval CLI into phase modules with typed seams (no-changelog) (#34691)

2026-07-22 16:25:44 +00:00

write-partial-results.test.ts

refactor(ai-builder): Split the eval harness runner into domain modules (no-changelog) (#34834)

2026-07-27 09:01:09 +00:00