opensource/n8n
3
0
Fork 0
mirror of https://github.com/n8n-io/n8n.git synced 2026-11-03 23:00:11 +08:00
Code Issues Packages Projects Releases Wiki Activity
21,941 Commits 2,556 Branches 2,241 Tags
865142f5da17035c243b14e2eba29bcd00867f50
Commit Graph
68 Commits
Author SHA1 Message Date
Benjamin Schrothandcubic-dev-ai[bot] d6dcf3b749 test(ai-builder): Support evals for agent building and for config evals (no-changelog) (#33888) 
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
 2026-07-15 12:34:21 +00:00
José Braulio González ValidoandClaude Fable 5 ced861603b test(ai-builder): Move the workflow-eval case corpus to LangTracer (no-changelog) (#33983) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-15 08:58:49 +00:00
José Braulio González ValidoandClaude Fable 5 2233d720ae ci: Bound and instrument Instance AI eval lane containers (no-changelog) (#33903) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-14 09:28:46 +00:00
José Braulio González ValidoandClaude Fable 5 3bc2292c78 test(ai-builder): Treat answer-only cases as valid without a built workflow (no-changelog) (#33942) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-10 21:54:14 +00:00
José Braulio González ValidoandClaude Fable 5 7dd065a77b feat(ai-builder): Scale eval budgets by case complexity and bound the heaviest prompts (no-changelog) (#33950) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-10 12:58:17 +00:00
José Braulio González ValidoandClaude Fable 5 8e6018b0a5 fix(ai-builder): Return experimentUrl from the empty-selection eval early exit (no-changelog) (#33947) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-10 12:50:43 +00:00
José Braulio González ValidoandClaude Fable 5 35632ce3ab fix(ai-builder): Sync execution scenarios on lang-tracer push updates (no-changelog) (#33841) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-10 12:50:12 +00:00
José Braulio González ValidoandClaude Fable 5 a88231544a feat(ai-builder): Persist eval expectation verdicts to LangSmith run outputs (no-changelog) (#33788) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-09 22:21:46 +00:00
Arvin AClaude Opus 4.8José Braulio González ValidoJosé Braulio González Valido
 		e6dee743e6 test(ai-builder): Add advanced user-behavior multi-turn eval cases (no-changelog) (#31758) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: José Braulio González Valido <jose.gonzalez@n8n.io>
Co-authored-by: José Braulio González Valido <josebragv@gmail.com>
 2026-07-09 18:13:42 +00:00
Mutasem AldmourandClaude Opus 4.8 c79184b5a5 feat(core): Add eval:langtracer-push to push eval cases to a suite (no-changelog) (#33781) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-08 07:52:23 +00:00
Robin BraumannandCursor f25699fe2c test(core): Extend Instance AI intent eval fixtures (no-changelog) (#33224) 
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-07-07 12:49:32 +00:00
José Braulio González ValidoandClaude Fable 5 a89f85b052 fix(ai-builder): Make eval verifier resilient to stalls and exclude no-verdict runs from scoring (no-changelog) (#33562) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-07 08:29:53 +00:00
Milorad FIlipović c5d65998da test: Reduce non-builder noise in MCP workflow evals (no-changelog) (#33565) 2026-07-06 13:48:11 +00:00
Albert Alises 2be4092c4d fix(core): Improve Instance AI workflow-building reliability (#33530) 2026-07-03 13:27:45 +00:00
Milorad FIlipović 579656db2e test: Parallelize MCP workflow evals across build lanes (no-changelog) (#33468) 2026-07-03 08:33:42 +00:00
oleg 6f0d30fbb1 fix: Improve AI Assistant post-build follow-ups (#33419) 2026-07-02 11:34:21 +00:00
Milorad FIlipović 3f3b3a756c fix(core): Harden MCP eval execution against transient network errors (no-changelog) (#33398) 2026-07-02 08:51:49 +00:00
José Braulio González ValidoandClaude Opus 4.8 dbadc276a2 test(ai-builder): Pin eval experiments to the Staging LangSmith workspace (no-changelog) (#33397) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-02 07:17:26 +00:00
Tomi Turtiainen 7fb04cdb98 refactor: Remove @n8n/utils barrel export (no-changelog) (#33082) 2026-06-30 16:05:24 +03:00
José Braulio González ValidoandClaude Opus 4.8 65f375bbd4 feat(ai-builder): Make eval executionScenarios optional for build-only cases (no-changelog) (#33252) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-30 09:41:14 +00:00
José Braulio González ValidoandClaude Opus 4.8 1d28330d18 feat(ai-builder): Dual-tenant LangSmith reads for Instance AI evals (no-changelog) (#33230) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-29 17:17:14 +00:00
José Braulio González ValidoandClaude Opus 4.8 23d70d1bb9 feat(ai-builder): Source eval test cases from LangTracer (no-changelog) (#33067) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-29 10:17:49 +00:00
Arvin A 9b7451f387 ci: Record git provenance metadata on eval experiments (no-changelog) (#33024) 2026-06-29 09:12:31 +00:00
José Braulio González ValidoandClaude Opus 4.8 c3741ce12a feat(ai-builder): Re-run Instance AI evals against the PR head on demand (no-changelog) (#33148) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-29 06:00:43 +00:00
José Braulio González ValidoandClaude Opus 4.8 d78aba164e test(ai-builder): Add absolute green-gate verdict for pr-tier evals (no-changelog) (#32984) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-26 10:00:06 +00:00
Milorad FIlipović 5b77a717b0 fix(core): Handle array-style conversations in mcp evals (no-changelog) (#33013) 2026-06-25 11:53:57 +00:00
Milorad FIlipović e12ce0ead0 feat(core): Enable LangSmith for mcp evaluations (no-changelog) (#32995) 2026-06-25 09:33:22 +00:00
Milorad FIlipović d3dd105aae feat(core): Make instanceAI evals more mcp-friendly (no-changelog) (#32916) 2026-06-25 06:33:50 +00:00
José Braulio González ValidoandClaude Fable 5 fe11536df1 feat(ai-builder): Add conversation pre-seeding to workflow evals (no-changelog) (#32196) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-06-23 17:30:30 +00:00
José Braulio González ValidoandClaude Fable 5 13239d1cfe feat(ai-builder): Add per-test-case eval credentials via thread credential-view pinning (no-changelog) (#32161) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-06-22 14:02:57 +00:00
yehorkardash d501854409 fix(core): Deduplicate isRecord (no-changelog) (#32469) 2026-06-22 08:35:03 +00:00
Riqwan Thamir b60bc923d4 feat(core): Add ability to view agent traces locally under a flag (no-changelog) (#32397) 2026-06-17 08:36:38 +00:00
José Braulio González ValidoandClaude Opus 4.8 5fa988104b test(ai-builder): Richer multi-turn eval framework (no-changelog) (#32054) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-11 12:57:08 +00:00
Milorad FIlipović 6166a1a94d chore(core): Respect buildCwd when discovering claude config in evaluations (no-changelog) (#31955) 2026-06-09 15:16:30 +00:00
oleg 77b95c611d feat(core): Harden Instance AI eval attribution and diagnostics (no-changelog) (#31951) 2026-06-09 12:10:10 +00:00
José Braulio González ValidoandClaude Opus 4.8 7791a940dd feat(ai-builder): Author-level conversation expectations on eval test cases (no-changelog) (#31787) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-09 10:06:07 +00:00
Milorad FIlipović 9bb77cc55e feat(core): Allow deleting pre-built workflows after evaluating (no-changelog) (#31960) 2026-06-09 08:50:31 +00:00
olegandRiqwan Thamir 52a1a0809a feat(core): Decouple Instance AI workflow verification from planning (#31340) 
Co-authored-by: Riqwan Thamir <rmthamir@gmail.com>
 2026-06-05 10:44:47 +00:00
José Braulio González ValidoandClaude Opus 4.7 46a222a15b ci: Re-enable Instance AI evals on PRs with cost-bounded trigger (no-changelog) (#31430) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-06-05 08:40:23 +00:00
Bernhard WittmannandRoman Davydchuk f4e998f245 fix(core): Prefer MCP registry nodes when wiring AI Agent tools (#30774) 
Co-authored-by: Roman Davydchuk <roman.davydchuk@n8n.io>
 2026-06-04 12:24:13 +00:00
José Braulio González ValidoandClaude Opus 4.8 f25e12de87 ci(ai-builder): Add iterations + experiment-name inputs to instance-ai eval dispatch (no-changelog) (#31631) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-04 09:57:08 +00:00
Albert Alises 332d2df44e fix(core): Move instance AI builds to a skill (#31412) 2026-06-02 16:06:00 +00:00
José Braulio González ValidoandClaude Opus 4.7 2a1f3d6b2d feat(ai-builder): Add datasets field for PR-tier suite (no-changelog) (#31429) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-06-02 13:30:31 +00:00
José Braulio González ValidoandClaude Opus 4.7 64f25b2ad8 feat(core): Show eval-mock executions in canvas list (no-changelog) (#31244) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-06-02 13:14:51 +00:00
Milorad FIlipović 9087a5ac6d fix(core): Fix multi-turn evals for mcp (no-changelog) (#31470) 2026-06-02 07:33:08 +00:00
Raúl Gómez Morales 24e95b164a refactor(core): Share scrubSecretsInText via @n8n/utils (no-changelog) (#31273) 2026-05-29 14:49:24 +00:00
Arvin A 55d8b59a48 feat(core): Stream tool calls and ship M3 fixtures from LLM eval wire server (no-changelog) (#30983) 2026-05-27 14:53:43 +01:00
Albert Alises 8bb5db3bbd feat(core): Add runtime skills to Instance AI builders (no-changelog) (#30838) 2026-05-27 06:54:40 +00:00
José Braulio González ValidoandClaude Opus 4.7 700b32237f feat(ai-builder): Surface WHAT-dimension binary checks per built workflow (no-changelog) (#30932) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-26 12:18:52 +01:00
José Braulio González ValidoandClaude Opus 4.7 96a9521394 ci: Use PR head ref in eval experiment names (no-changelog) (#30898) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-22 11:06:50 +00:00