Commit Graph
70 Commits
Author SHA1 Message Date
oleg 97446258af feat(editor): Collapse AI Assistant thinking traces into thinking blocks (#34274)
Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>
2026-07-16 08:56:19 +00:00
Benjamin Schrothandcubic-dev-ai[bot] d6dcf3b749 test(ai-builder): Support evals for agent building and for config evals (no-changelog) (#33888)
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
2026-07-15 12:34:21 +00:00
José Braulio González ValidoandClaude Fable 5 2233d720ae ci: Bound and instrument Instance AI eval lane containers (no-changelog) (#33903)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 09:28:46 +00:00
Filipe TavaresandClaude Opus 4.8 67b3ed11fe feat(editor): Start Instance AI threads from templates on the website and template page (#33653)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 15:41:54 +00:00
Robin BraumannandCursor dc5cc1434e chore(core): Remove the assistant's embedded agent-builder toolset (no-changelog) (#34066)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-13 14:20:13 +00:00
José Braulio González ValidoandClaude Fable 5 3bc2292c78 test(ai-builder): Treat answer-only cases as valid without a built workflow (no-changelog) (#33942)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 21:54:14 +00:00
José Braulio González ValidoandClaude Fable 5 13c95f2c05 fix(ai-builder): Eliminate non-builder noise families in workflow evals (no-changelog) (#33944)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 13:25:29 +00:00
José Braulio González ValidoandClaude Fable 5 7dd065a77b feat(ai-builder): Scale eval budgets by case complexity and bound the heaviest prompts (no-changelog) (#33950)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 12:58:17 +00:00
José Braulio González ValidoandClaude Opus 4.8 1d61db2252 test(ai-builder): Rename response-matches eval check and fix multi-turn wiring (no-changelog) (#32810)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 12:58:13 +00:00
e6dee743e6 test(ai-builder): Add advanced user-behavior multi-turn eval cases (no-changelog) (#31758)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: José Braulio González Valido <jose.gonzalez@n8n.io>
Co-authored-by: José Braulio González Valido <josebragv@gmail.com>
2026-07-09 18:13:42 +00:00
Jaakko HussoandClaude Fable 5 4dd369991f feat(core): Use a shared per-thread event sequence for Instance AI multi-main (no-changelog) (#33558)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-09 07:44:20 +00:00
5c6544af4a feat(core): Port agent builder skills and tools to Instance AI (no-changelog) (#33384)
Co-authored-by: Robin Braumann <robin.braumann@n8n.io>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: bjorger <50590409+bjorger@users.noreply.github.com>
Co-authored-by: Michael Drury <michael.drury@n8n.io>
Co-authored-by: Michael Drury <me@michaeldrury.co.uk>
2026-07-08 12:41:12 +00:00
Mutasem AldmourandClaude Opus 4.8 c79184b5a5 feat(core): Add eval:langtracer-push to push eval cases to a suite (no-changelog) (#33781)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-08 07:52:23 +00:00
Robin BraumannandCursor f25699fe2c test(core): Extend Instance AI intent eval fixtures (no-changelog) (#33224)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-07 12:49:32 +00:00
José Braulio González ValidoandClaude Fable 5 a89f85b052 fix(ai-builder): Make eval verifier resilient to stalls and exclude no-verdict runs from scoring (no-changelog) (#33562)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-07 08:29:53 +00:00
José Braulio González ValidoandClaude Opus 4.8 eb59191076 feat(ai-builder): Capture compiled workflow JSON as a trace event for eval seed reconstruction (no-changelog) (#33302)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 15:03:38 +00:00
Milorad FIlipović c5d65998da test: Reduce non-builder noise in MCP workflow evals (no-changelog) (#33565) 2026-07-06 13:48:11 +00:00
Albert Alises 2be4092c4d fix(core): Improve Instance AI workflow-building reliability (#33530) 2026-07-03 13:27:45 +00:00
Milorad FIlipović 579656db2e test: Parallelize MCP workflow evals across build lanes (no-changelog) (#33468) 2026-07-03 08:33:42 +00:00
oleg 6f0d30fbb1 fix: Improve AI Assistant post-build follow-ups (#33419) 2026-07-02 11:34:21 +00:00
Milorad FIlipović 3f3b3a756c fix(core): Harden MCP eval execution against transient network errors (no-changelog) (#33398) 2026-07-02 08:51:49 +00:00
José Braulio González ValidoandClaude Opus 4.8 dbadc276a2 test(ai-builder): Pin eval experiments to the Staging LangSmith workspace (no-changelog) (#33397)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 07:17:26 +00:00
Tomi Turtiainen 7fb04cdb98 refactor: Remove @n8n/utils barrel export (no-changelog) (#33082) 2026-06-30 16:05:24 +03:00
José Braulio González ValidoandClaude Opus 4.8 65f375bbd4 feat(ai-builder): Make eval executionScenarios optional for build-only cases (no-changelog) (#33252)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 09:41:14 +00:00
José Braulio González ValidoandClaude Opus 4.8 7ea6900e58 feat(ai-builder): Honor seedThread.liveTurnRunId in eval reconstruction (no-changelog) (#33251)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 09:34:10 +00:00
José Braulio González ValidoandClaude Opus 4.8 1d28330d18 feat(ai-builder): Dual-tenant LangSmith reads for Instance AI evals (no-changelog) (#33230)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 17:17:14 +00:00
José Braulio González ValidoandClaude Opus 4.8 23d70d1bb9 feat(ai-builder): Source eval test cases from LangTracer (no-changelog) (#33067)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 10:17:49 +00:00
José Braulio González ValidoandClaude Opus 4.8 3380ca3790 fix(ai-builder): Reconstruct file-based seeded workflows and judge the whole conversation (no-changelog) (#33048)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 10:06:20 +00:00
José Braulio González ValidoandClaude Opus 4.8 974e7be09b fix(ai-builder): Scope seedThread reconstruction and add per-turn tool counts for the judge (no-changelog) (#32962)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-25 10:56:59 +00:00
Milorad FIlipović d3dd105aae feat(core): Make instanceAI evals more mcp-friendly (no-changelog) (#32916) 2026-06-25 06:33:50 +00:00
José Braulio González ValidoandClaude Opus 4.8 d7a331afd4 test(ai-builder): Detect excessive build-workflow failures in instance-ai evals (no-changelog) (#32808)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 09:56:59 +00:00
José Braulio González ValidoandClaude Fable 5 fe11536df1 feat(ai-builder): Add conversation pre-seeding to workflow evals (no-changelog) (#32196)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-06-23 17:30:30 +00:00
José Braulio González ValidoandClaude Fable 5 13239d1cfe feat(ai-builder): Add per-test-case eval credentials via thread credential-view pinning (no-changelog) (#32161)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-06-22 14:02:57 +00:00
Riqwan ThamirandCursor c7143e0e17 feat(core): Await memory tasks when running agent through evals (no-changelog) (#32693)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-22 11:11:56 +00:00
yehorkardash d501854409 fix(core): Deduplicate isRecord (no-changelog) (#32469) 2026-06-22 08:35:03 +00:00
Riqwan Thamir b60bc923d4 feat(core): Add ability to view agent traces locally under a flag (no-changelog) (#32397) 2026-06-17 08:36:38 +00:00
José Braulio González ValidoandClaude Fable 5 4573969629 fix(ai-builder): Harden eval mock layer for text responses, trigger binary and lookup attribution (no-changelog) (#32179)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-06-15 11:04:59 +00:00
José Braulio González ValidoandClaude Opus 4.8 5fa988104b test(ai-builder): Richer multi-turn eval framework (no-changelog) (#32054)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 12:57:08 +00:00
oleg 77b95c611d feat(core): Harden Instance AI eval attribution and diagnostics (no-changelog) (#31951) 2026-06-09 12:10:10 +00:00
José Braulio González ValidoandClaude Opus 4.8 7791a940dd feat(ai-builder): Author-level conversation expectations on eval test cases (no-changelog) (#31787)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 10:06:07 +00:00
Milorad FIlipović 9bb77cc55e feat(core): Allow deleting pre-built workflows after evaluating (no-changelog) (#31960) 2026-06-09 08:50:31 +00:00
Albert Alises 4271d42398 fix(core): Teach the builder the SDK language subset and stop repeated invalid repairs (#31911) 2026-06-09 07:42:28 +00:00
oleg 2925d654d1 fix: Use UUID thread IDs for Instance AI evals (no-changelog) (#31830) 2026-06-05 13:15:45 +00:00
Albert Alises 6cd07e5b42 fix: Add item-flow builder guidance (no-changelog) (#31660) 2026-06-04 09:23:20 +00:00
olegandClaude afd7ddf372 feat(core): Use n8n default sandbox for Instance AI (no-changelog) (#31335)
Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-06-04 08:31:27 +00:00
Raúl Gómez Morales e27c4feaca feat(core): Invalidate instance-ai build-workflow cache on canvas edits (#31274) 2026-06-03 11:09:23 +00:00
Albert Alises 332d2df44e fix(core): Move instance AI builds to a skill (#31412) 2026-06-02 16:06:00 +00:00
José Braulio González ValidoandClaude Opus 4.7 64f25b2ad8 feat(core): Show eval-mock executions in canvas list (no-changelog) (#31244)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-02 13:14:51 +00:00
José Braulio González ValidoandClaude Opus 4.7 a1369e7736 fix(core): Preserve execution output fidelity in eval verifier artifact (no-changelog) (#30989)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-02 08:55:01 +00:00
Arvin A 55d8b59a48 feat(core): Stream tool calls and ship M3 fixtures from LLM eval wire server (no-changelog) (#30983) 2026-05-27 14:53:43 +01:00