From 2c79d0249f1cabe1c4cb0c327e5d9ccf24d83a2f Mon Sep 17 00:00:00 2001 From: Waleed Date: Tue, 3 Mar 2026 19:21:52 -0800 Subject: [PATCH] improvement(executor): support nested loops/parallels (#3398) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(executor): support nested loop DAG construction and edge wiring Wire inner loop sentinel nodes into outer loop sentinel chains so that nested loops execute correctly. Resolves boundary-node detection to use effective sentinel IDs for nested loops, handles loop-exit edges from inner sentinel-end to outer sentinel-end, and recursively clears execution state for all nested loop scopes between iterations. NOTE: loop-in-loop nesting only; parallel nesting is not yet supported. Made-with: Cursor * feat(executor): add nested loop iteration context and named loop variable resolution Introduce ParentIteration to track ancestor loop state, build a loopParentMap during DAG construction, and propagate parent iterations through block execution and child workflow contexts. Extend LoopResolver to support named loop references (e.g. ) and add output property resolution (). Named references use the block's display name normalized to a tag-safe identifier, enabling blocks inside nested loops to reference any ancestor loop's iteration state. NOTE: loop-in-loop nesting only; parallel nesting is not yet supported. Made-with: Cursor * feat(terminal): propagate parent iteration context through SSE events and terminal display Thread parentIterations through SSE block-started, block-completed, and block-error events so the terminal can reconstruct nested loop hierarchies. Update the entry tree builder to recursively nest inner loop subflow nodes inside their parent iteration rows, using parentIterations depth-stripping to support arbitrary nesting depth. Display the block's store name for subflow container rows instead of the generic "Loop" / "Parallel" label. Made-with: Cursor * feat(canvas): allow nesting subflow containers and prevent cycles Remove the restriction that prevented subflow nodes from being dragged into other subflow containers, enabling loop-in-loop nesting on the canvas. Add cycle detection (isDescendantOf) to prevent a container from being placed inside one of its own descendants. Resize all ancestor containers when a nested child moves, collect descendant blocks when removing from a subflow so boundary edges are attributed correctly, and surface all ancestor loop tags in the tag dropdown for blocks inside nested loops. Made-with: Cursor * feat(agent): add MCP server discovery mode for agent tool input (#3353) * feat(agent): add MCP server discovery mode for agent tool input * fix(tool-input): use type variant for MCP server tool count badge * fix(mcp-dynamic-args): align label styling with standard subblock labels * standardized inp format UI * feat(tool-input): replace MCP server inline expand with drill-down navigation * feat(tool-input): add chevron affordance and keyboard nav for MCP server drill-down * fix(tool-input): handle mcp-server type in refresh, validation, badges, and usage control * refactor(tool-validation): extract getMcpServerIssue, remove fake tool hack * lint * reorder dropdown * perf(agent): parallelize MCP server tool creation with Promise.all * fix(combobox): preserve cursor movement in search input, reset query on drilldown * fix(combobox): route ArrowRight through handleSelect, remove redundant type guards * fix(agent): rename mcpServers to mcpServerSelections to avoid shadowing DB import, route ArrowRight through handleSelect * docs: update google integration docs * fix(tool-input): reset drilldown state on tool selection to prevent stale view * perf(agent): parallelize MCP server discovery across multiple servers * improvement(tests): speed up unit tests by eliminating vi.resetModules anti-pattern (#3357) * improvement(tests): speed up unit tests by eliminating vi.resetModules anti-pattern - convert 51 test files from vi.resetModules/vi.doMock/dynamic import to vi.hoisted/vi.mock/static import - add global @sim/db mock to vitest.setup.ts - switch 4 test files from jsdom to node environment - remove all vi.importActual calls that loaded heavy modules (200+ block files) - remove slow mockConsoleLogger/mockAuth/setupCommonApiMocks helpers - reduce real setTimeout delays in engine tests - mock heavy transitive deps in diff-engine test test execution time: 34s -> 9s (3.9x faster) environment time: 2.5s -> 0.6s (4x faster) * docs(testing): update testing best practices with performance rules - document vi.hoisted + vi.mock + static import as the standard pattern - explicitly ban vi.resetModules, vi.doMock, vi.importActual, mockAuth, setupCommonApiMocks - document global mocks from vitest.setup.ts - add mock pattern reference for auth, hybrid auth, and database chains - add performance rules section covering heavy deps, jsdom vs node, real timers * fix(tests): fix 4 failing test files with missing mocks - socket/middleware/permissions: add vi.mock for @/lib/auth to prevent transitive getBaseUrl() call - workflow-handler: add vi.mock for @/executor/utils/http matching executor mock pattern - evaluator-handler: add db.query.account mock structure before vi.spyOn - router-handler: same db.query.account fix as evaluator * fix(tests): replace banned Function type with explicit callback signature * feat(databricks): add Databricks integration with 8 tools (#3361) * feat(databricks): add Databricks integration with 8 tools Add complete Databricks integration supporting SQL execution, job management, run monitoring, and cluster listing via Personal Access Token authentication. Tools: execute_sql, list_jobs, run_job, get_run, list_runs, cancel_run, get_run_output, list_clusters Co-Authored-By: Claude Opus 4.6 * fix(databricks): throw on invalid JSON params, fix boolean coercion, add expandTasks field - Throw errors on invalid JSON in jobParameters/notebookParams instead of silently defaulting to {} - Always set boolean params explicitly to prevent string 'false' being truthy - Add missing expandTasks dropdown UI field for list_jobs operation * fix(databricks): align tool inputs/outputs with official API spec - execute_sql: fix wait_timeout default description (50s, not 10s) - get_run: add queueDuration field, update lifecycle/result state enums - get_run_output: fix notebook output size (5 MB not 1 MB), add logsTruncated field - list_runs: add userCancelledOrTimedout to state, fix limit range (1-24), update state enums - list_jobs: fix name filter description to "exact case-insensitive" - list_clusters: add PIPELINE_MAINTENANCE to ClusterSource enum * fix(databricks): regenerate docs to reflect API spec fixes --------- Co-authored-by: Claude Opus 4.6 * feat(luma): add Luma integration for event and guest management (#3364) * feat(luma): add Luma integration for event and guest management Add complete Luma (lu.ma) integration with 6 tools: get event, create event, update event, list calendar events, get guests, and add guests. Includes block configuration with wandConfig for timestamps/timezones/durations, advanced mode for optional fields, and generated documentation. Co-Authored-By: Claude Opus 4.6 * fix(luma): address PR review feedback - Remove hosts field from list_events transformResponse (not in LumaEventEntry type) - Fix truncated add_guests description by removing quotes that broke docs generator Co-Authored-By: Claude Opus 4.6 * fix(luma): fix update_event field name and add_guests response parsing - Use 'id' instead of 'event_id' in update_event request body per API spec - Fix add_guests to parse entries[].guest response structure instead of flat guests array Co-Authored-By: Claude Opus 4.6 --------- Co-authored-by: Claude Opus 4.6 * feat(gamma): add gamma integration for AI-powered content generation (#3358) * feat(gamma): add gamma integration for AI-powered content generation * fix(gamma): address PR review comments - Make credits/error conditionally included in check_status response to avoid always-truthy objects - Replace full wordmark SVG with square "G" letterform for proper rendering in icon slots * fix(gamma): remove imageSource from generate_from_template endpoint The from-template API only accepts imageOptions.model and imageOptions.style, not imageOptions.source (image source is inherited from the template). * fix(gamma): use typed output in check_status transformResponse * regen docs * feat(greenhouse): add greenhouse integration for managing candidates, jobs, and applications (#3363) * feat(ashby): add ashby integration for candidate, job, and application management (#3362) * feat(ashby): add ashby integration for candidate, job, and application management * fix(ashby): auto-fix lint formatting in docs files * improvement(oauth): reordered oauth modal (#3368) * feat(loops): add Loops email platform integration (#3359) * feat(loops): add Loops email platform integration Add complete Loops integration with 10 tools covering all API endpoints: - Contact management: create, update, find, delete - Email: send transactional emails with attachments - Events: trigger automated email sequences - Lists: list mailing lists and transactional email templates - Properties: create and list contact properties Co-Authored-By: Claude Opus 4.6 * ran litn --------- Co-authored-by: Claude Opus 4.6 * feat(resend): expand integration with contacts, domains, and enhanced email ops (#3366) * improvement(blocks): update luma styling and linkup field modes (#3370) * improvement(blocks): update luma styling and linkup field modes * improvement(fireflies): move optional fields to advanced mode * improvement(blocks): move optional fields to advanced mode for 10 integrations * improvement(blocks): move optional fields to advanced mode for 6 more integrations * feat(x): add 28 new X API v2 tool integrations and expand OAuth scopes (#3365) * feat(x): add 28 new X API v2 tool integrations and expand OAuth scopes * fix(x): add missing nextToken param to search tweets and fix XCreateTweetParams type * fix(x): correct API spec issues in retweeted_by, quote_tweets, personalized_trends, and usage tools * fix(x): add missing newestId and oldestId to error meta in get_liked_tweets and get_quote_tweets * fix(x): add missing newestId/oldestId to get_liked_tweets success branch and includes to XTweetListResponse * fix(x): add error handling to create_tweet and delete_tweet transformResponse * fix(x): add error handling and logger to all X tools * fix(x): revert block requiredScopes to match current operations * feat(x): update block to support all 28 new X API v2 tools * fix(x): add missing text output and fix hiddenResult output key mismatch * docs(x): regenerate docs for all 28 new X API v2 tools * improvement(docs): audit and standardize tool description sections, update developer count to 70k (#3371) * improvement(x): align OAuth scopes, add scope descriptions, and set optional fields to advanced mode (#3372) * improvement(x): align OAuth scopes, add scope descriptions, and set optional fields to advanced mode * improvement(skills): add typed JSON outputs guidance to add-tools, add-block, and add-integration skills * improvement(skills): add final validation steps to add-tools, add-block, and add-integration skills * fix(skills): correct misleading JSON array comment in wandConfig example * feat(skills): add validate-integration skill for auditing tools, blocks, and registry against API docs * improvement(skills): expand validate-integration with full block-tool alignment, OAuth scopes, pagination, and error handling checks * improvement(ci): add sticky disk caches and bump runner for faster builds (#3373) * improvement(selectors): make selectorKeys declarative (#3374) * fix(webflow): resolution for selectors * remove unecessary fallback' * fix teams selector resolution * make selector keys declarative * selectors fixes * improvement(selectors): consolidate selector input logic (#3375) * feat(google-contacts): add google contacts integration (#3340) * feat(google-contacts): add google contacts integration * fix(google-contacts): throw error when no update fields provided * lint * update icon * improvement(google-contacts): add advanced mode, error handling, and input trimming - Set mode: 'advanced' on optional fields (emailType, phoneType, notes, pageSize, pageToken, sortOrder) - Add createLogger and response.ok error handling to all 6 tools - Add .trim() on resourceName in get, update, delete URL builders * improvement(mcp): add all MCP server tools individually instead of as single server entry (#3376) * improvement(mcp): add all MCP server tools individually instead of as single server entry * fix(mcp): prevent remove popover from opening inadvertently * fix(sse): fix memory leaks in SSE stream cleanup and add memory telemetry (#3378) * fix(sse): fix memory leaks in SSE stream cleanup and add memory telemetry * improvement(monitoring): add SSE metering to wand, execution-stream, and a2a-message endpoints * fix(workflow-execute): remove abort from cancel() to preserve run-on-leave behavior * improvement(monitoring): use stable process.getActiveResourcesInfo() API * refactor(a2a): hoist resubscribe cleanup to eliminate duplication between start() and cancel() * style(a2a): format import line Co-Authored-By: Claude Opus 4.6 * fix(wand): set guard flag on early-return decrement for consistency --------- Co-authored-by: Claude Opus 4.6 * improvement(ashby): validate ashby integration and update skill files (#3381) * improvement(luma): expand host response fields and harden event ID inputs (#3383) * improvement(resend): add error handling, authMode, and naming consistency (#3382) * fix(chat-deploy): fix launch chat popup and auth persistence, clean up React anti-patterns (#3380) * fix(chat-deploy): fix launch chat popup and auth persistence, clean up React anti-patterns * lint * fix(greenhouse): fix email_address query param, add .trim() to ID paths, revert onValidationChange to useEffect * fix(chat-deploy): fix stale AuthSelector state, stabilize refetch ref, clean up copy timeout * fix(chat-deploy): reset chatSuccess on modal open to prevent stuck state * improvement(loops): validate loops integration and update skill files (#3384) * improvement(loops): validate loops integration and update skill files * loops icon color * update databricks icon * fix(monitoring): set MemoryTelemetry logger to INFO level for production visibility (#3386) Production defaults to ERROR-only logging. Without this override, memory snapshots would be silently suppressed. * feat(integrations): add amplitude, google pagespeed insights, and pagerduty integrations (#3385) * feat(integrations): add amplitude and google pagespeed insights integrations * verified and regen docs * fix icons * fix(integrations): add pagerduty to tool and block registries Re-add registry entries that were reverted after initial commit. Co-Authored-By: Claude Opus 4.6 * more updates * ack comemnts --------- Co-authored-by: Claude Opus 4.6 * feat(docs): add API reference with OpenAPI spec and auto-generated endpoint pages (#3388) * feat(docs): add API reference with OpenAPI spec and auto-generated endpoint pages * multiline curl * random improvements * cleanup * update docs copy * fix build * cast * fix builg --------- Co-authored-by: Claude Opus 4.6 Co-authored-by: Lakee Sivaraya <71339072+lakeesiv@users.noreply.github.com> Co-authored-by: Vikhyath Mondreti Co-authored-by: Vikhyath Mondreti * fix(icons): fix pagerduty icon (#3392) * improvement(executor): audit and harden nested loop/parallel implementation * improvement(executor): audit and harden nested loop/parallel implementation - Replace unsafe _childWorkflowInstanceId cast with typeof type guard - Reuse WorkflowNodeMetadata interface instead of inline type duplication - Rename _executeCore to executeCore (private, no underscore needed) - Add log warning when SSE callbacks are dropped beyond MAX_SSE_CHILD_DEPTH - Remove unnecessary onStream type assertion, use StreamingExecution type - Convert OUTPUT_PROPERTIES/KNOWN_PROPERTIES from arrays to Sets for O(1) lookup - Add type guard in loop resolver resolveOutput before casting - Add TSDoc to edgeCrossesLoopBoundary explaining original-ID usage - Add TSDoc to MAX_SSE_CHILD_DEPTH constant - Update ParentIteration TSDoc to reflect parallel nesting support - Type usageControl as union 'auto'|'force'|'none' in buildMcpTool - Replace (t: any) casts with typed objects in agent-handler tests - Add type guard in builder-data convertArrayItem - Make ctx required in clearLoopExecutionState (only caller always passes it) - Replace Math.random() with deterministic counter in terminal tests - Fix isWorkflowBlockType mock to actually check block types - Add loop-in-loop and workflow block tree tests * improvement(executor): audit fixes for nested subflow implementation - Fix findInnermostLoopForBlock/ParallelForBlock to return deepest nested container instead of first Object.keys() match - Fix isBlockInLoopOrDescendant returning false when directLoopId equals target (should return true) - Add isBlockInParallelOrDescendant with recursive nested parallel checking to match loop resolver behavior - Extract duplicated ~20-line iteration context building from loop/parallel orchestrators into shared buildContainerIterationContext utility - Remove inline import() type references in orchestrators - Remove dead executionOrder field from WorkflowNodeMetadata - Remove redundant double-normalization in findParallelBoundaryNodes - Consolidate 3 identical tree-walk helpers into generic hasMatchInTree - Add empty-array guards for Math.min/Math.max in terminal utils - Make KNOWN_PROPERTIES a Set in parallel resolver for consistency - Remove no-op handleDragEnd callback from toolbar - Remove dead result/results entries from KNOWN_PROPERTIES in loop resolver - Add tests for buildContainerIterationContext Co-Authored-By: Claude Opus 4.6 * finished * improvement(airtable): added more tools (#3396) * fix(layout): polyfill crypto.randomUUID for non-secure HTTP contexts (#3397) * feat(integrations): add dub.co integration (#3400) * feat(integrations): add dub.co integration * improvement(dub): add manual docs description and lint formatting fixes * lint * fix(dub): remove unsupported optional property from block outputs * fix(memory): fix O(n²) string concatenation and unconsumed fetch response leaks (#3399) * fix(monitoring): set MemoryTelemetry logger to INFO level for production visibility Production defaults to ERROR-only logging. Without this override, memory snapshots would be silently suppressed. * fix(memory): fix O(n²) string concatenation and unconsumed fetch response leaks * fix(tests): add text() mock to workflow-handler test fetch responses * fix(memory): remove unused O(n²) join in onStreamChunk callback * chore(careers): remove careers page, redirect to Ashby jobs portal (#3401) * chore(careers): remove careers page, redirect to Ashby jobs portal * lint * feat(integrations): add google meet integration (#3403) * feat(integrations): add google meet integration * lint * ack comments * ack comments * fix(terminal): deduplicate nested container entries in buildEntryTree Filter out container-typed block rows when matching nested subflow nodes exist, preventing nested loops/parallels from appearing twice (once as a flat block and once as an expandable subflow). Co-Authored-By: Claude Opus 4.6 * improvement(executor): clean up nested subflow implementation - Fix wireSentinelEdges to use LOOP_EXIT handle for nested loop terminals - Extract buildExecutionPipeline to deduplicate orchestrator wiring - Replace two-phase init with constructor injection for Loop/ParallelOrchestrator - Remove dead code: shouldExecuteLoopNode, resolveForEachItems, isLoopNode, isParallelNode, isSubflowBlockType - Deduplicate currentItem resolution in ParallelResolver via resolveCurrentItem - Type getDistributionItems param as SerializedParallel instead of any - Demote verbose per-reference logger.info to logger.debug in evaluateWhileCondition - Add loop-in-parallel wiring test in edges.test.ts Co-Authored-By: Claude Opus 4.6 * fix(test): update parallel resolver test to use distribution instead of distributionItems The distributionItems fallback was never part of SerializedParallel — it only worked through any typing. Updated the test to use the real distribution property. Co-Authored-By: Claude Opus 4.6 * fix(executor): skip loop back-edges in parallel boundary detection and update test findParallelBoundaryNodes now skips LOOP_CONTINUE back-edges when detecting terminal nodes, matching findLoopBoundaryNodes behavior. Without this, a nested loop's back-edge was incorrectly counted as a forward edge within the parallel, preventing terminal detection. Also updated parallel resolver test to use the real distribution property instead of the non-existent distributionItems fallback. Co-Authored-By: Claude Opus 4.6 * fix(executor): clean up cloned loop scopes in deleteParallelScopeAndClones When a parallel contains a nested loop, cloned loop scopes (__obranch-N) created by expandParallel were not being deleted, causing stale scopes to persist across outer loop iterations. Co-Authored-By: Claude Opus 4.6 * fix(executor): remove dead fallbacks, fix nested loop boundary detection, restore executionOrder - Remove unreachable `?? candidateIds[0]` fallbacks in loop/parallel resolvers - Remove arbitrary first-match fallback scan in findEffectiveContainerId - Fix edgeCrossesLoopBoundary to use innermost loop detection for nested loops - Add warning log for missing branch outputs in parallel aggregation - Restore executionOrder on WorkflowNodeMetadata and pipe through child workflow notification - Remove dead sim-drag-subflow classList.remove call - Clean up cloned loop subflowParentMap entries in deleteParallelScopeAndClones Co-Authored-By: Claude Opus 4.6 * leftover * upgrade turborepo * update stagehand icon * fix(tag-dropdown): show contextual loop/parallel tags for deeply nested blocks findAncestorLoops only checked direct loop membership, missing blocks nested inside parallels within loops (and vice versa). Refactored to walk through both loop and parallel containers recursively, so a block inside a parallel inside a loop correctly sees the loop's contextual tags (index, currentItem) instead of the loop's output tags (results). Also fixed parallel ancestor detection to handle nested parallel-in-loop and loop-in-parallel scenarios, collecting all ancestor parallels instead of just the immediate containing one. Co-Authored-By: Claude Opus 4.6 * testing * fixed dedicated logs * fix * fix(subflows): enable nested subflow interaction and execution highlighting Remove !important z-index overrides that prevented nested subflows from being grabbed/dragged independently. Z-index is now managed by ReactFlow's elevateNodesOnSelect and per-node zIndex: depth props. Also adds execution status highlighting for nested subflows in both canvas and snapshot preview. Co-Authored-By: Claude Opus 4.6 * fix(preview): add cycle guard to recursive subflow status derivation Prevents infinite recursion if subflowChildrenMap contains circular references by tracking visited nodes during traversal. Co-Authored-By: Claude Opus 4.6 --------- Co-authored-by: Claude Opus 4.6 Co-authored-by: Lakee Sivaraya <71339072+lakeesiv@users.noreply.github.com> Co-authored-by: Vikhyath Mondreti Co-authored-by: Vikhyath Mondreti Co-authored-by: Vasyl Abramovych --- apps/docs/components/icons.tsx | 164 +---- apps/sim/app/_styles/globals.css | 10 +- .../app/api/workflows/[id]/execute/route.ts | 9 + .../components/tag-dropdown/tag-dropdown.tsx | 182 +++--- .../panel/components/toolbar/toolbar.tsx | 10 - .../components/subflows/subflow-node.tsx | 74 ++- .../components/terminal/terminal.tsx | 51 +- .../components/terminal/utils.test.ts | 478 ++++++++++++++ .../[workflowId]/components/terminal/utils.ts | 305 +++++---- .../[workflowId]/hooks/use-node-utilities.ts | 34 +- .../hooks/use-workflow-execution.ts | 76 +-- .../utils/workflow-canvas-helpers.ts | 33 + .../[workspaceId]/w/[workflowId]/workflow.tsx | 261 +++++--- .../preview-workflow/preview-workflow.tsx | 33 +- apps/sim/components/icons.tsx | 164 +---- apps/sim/executor/constants.ts | 2 + apps/sim/executor/dag/builder.test.ts | 95 +++ apps/sim/executor/dag/builder.ts | 4 +- .../executor/dag/construction/edges.test.ts | 591 ++++++++++++++++++ apps/sim/executor/dag/construction/edges.ts | 279 +++++++-- apps/sim/executor/execution/block-executor.ts | 75 +-- apps/sim/executor/execution/executor.ts | 82 ++- apps/sim/executor/execution/types.ts | 46 +- .../handlers/agent/agent-handler.test.ts | 34 +- .../executor/handlers/agent/agent-handler.ts | 13 +- .../handlers/workflow/workflow-handler.ts | 81 +-- apps/sim/executor/human-in-the-loop/utils.ts | 8 +- apps/sim/executor/orchestrators/loop.ts | 290 +++++++-- apps/sim/executor/orchestrators/node.ts | 10 +- apps/sim/executor/orchestrators/parallel.ts | 105 +++- apps/sim/executor/types.ts | 15 + apps/sim/executor/utils/builder-data.ts | 51 +- .../executor/utils/iteration-context.test.ts | 585 +++++++++++++++++ apps/sim/executor/utils/iteration-context.ts | 179 ++++++ .../executor/utils/parallel-expansion.test.ts | 316 ++++++++++ apps/sim/executor/utils/parallel-expansion.ts | 290 ++++++++- apps/sim/executor/utils/subflow-utils.ts | 87 ++- .../executor/variables/resolvers/loop.test.ts | 168 ++++- apps/sim/executor/variables/resolvers/loop.ts | 187 +++++- .../variables/resolvers/parallel.test.ts | 8 +- .../executor/variables/resolvers/parallel.ts | 187 ++++-- .../execution/trace-spans/trace-spans.test.ts | 440 +++++++++++++ .../logs/execution/trace-spans/trace-spans.ts | 588 ++++++++--------- apps/sim/lib/logs/types.ts | 3 +- .../workflows/executor/execution-events.ts | 15 +- apps/sim/stores/terminal/console/store.ts | 4 + apps/sim/stores/terminal/console/types.ts | 3 + bun.lock | 16 +- package.json | 2 +- 49 files changed, 5299 insertions(+), 1444 deletions(-) create mode 100644 apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/terminal/utils.test.ts create mode 100644 apps/sim/executor/utils/iteration-context.test.ts create mode 100644 apps/sim/executor/utils/iteration-context.ts create mode 100644 apps/sim/executor/utils/parallel-expansion.test.ts diff --git a/apps/docs/components/icons.tsx b/apps/docs/components/icons.tsx index 22c5b5718e..5525e048cf 100644 --- a/apps/docs/components/icons.tsx +++ b/apps/docs/components/icons.tsx @@ -1711,167 +1711,31 @@ export function StagehandIcon(props: SVGProps) { return ( - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ) diff --git a/apps/sim/app/_styles/globals.css b/apps/sim/app/_styles/globals.css index f7e15a76cb..1a74ca50df 100644 --- a/apps/sim/app/_styles/globals.css +++ b/apps/sim/app/_styles/globals.css @@ -833,15 +833,7 @@ input[type="search"]::-ms-clear { animation: growShrink 1.5s infinite ease-in-out; } -/* Subflow node z-index and drag-over styles */ -.workflow-container .react-flow__node-subflowNode { - z-index: -1 !important; -} - -.workflow-container .react-flow__node-subflowNode:has([data-subflow-selected="true"]) { - z-index: 10 !important; -} - +/* Subflow node drag-over styles */ .loop-node-drag-over, .parallel-node-drag-over { box-shadow: 0 0 0 1.75px var(--brand-secondary) !important; diff --git a/apps/sim/app/api/workflows/[id]/execute/route.ts b/apps/sim/app/api/workflows/[id]/execute/route.ts index b2cb3c1f8c..ea7c792bb1 100644 --- a/apps/sim/app/api/workflows/[id]/execute/route.ts +++ b/apps/sim/app/api/workflows/[id]/execute/route.ts @@ -828,6 +828,9 @@ export async function POST(req: NextRequest, { params }: { params: Promise<{ id: iterationTotal: iterationContext.iterationTotal, iterationType: iterationContext.iterationType, iterationContainerId: iterationContext.iterationContainerId, + ...(iterationContext.parentIterations?.length && { + parentIterations: iterationContext.parentIterations, + }), }), ...(childWorkflowContext && { childWorkflowBlockId: childWorkflowContext.parentBlockId, @@ -884,6 +887,9 @@ export async function POST(req: NextRequest, { params }: { params: Promise<{ id: iterationTotal: iterationContext.iterationTotal, iterationType: iterationContext.iterationType, iterationContainerId: iterationContext.iterationContainerId, + ...(iterationContext.parentIterations?.length && { + parentIterations: iterationContext.parentIterations, + }), }), ...childWorkflowData, ...instanceData, @@ -915,6 +921,9 @@ export async function POST(req: NextRequest, { params }: { params: Promise<{ id: iterationTotal: iterationContext.iterationTotal, iterationType: iterationContext.iterationType, iterationContainerId: iterationContext.iterationContainerId, + ...(iterationContext.parentIterations?.length && { + parentIterations: iterationContext.parentIterations, + }), }), ...childWorkflowData, ...instanceData, diff --git a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/sub-block/components/tag-dropdown/tag-dropdown.tsx b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/sub-block/components/tag-dropdown/tag-dropdown.tsx index 0c1dbc951f..8fcdf62c1c 100644 --- a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/sub-block/components/tag-dropdown/tag-dropdown.tsx +++ b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/sub-block/components/tag-dropdown/tag-dropdown.tsx @@ -1167,21 +1167,56 @@ export const TagDropdown: React.FC = ({ {} as Record ) - let loopBlockGroup: BlockTagGroup | null = null + const loopBlockGroups: BlockTagGroup[] = [] + const ancestorLoopIds = new Set() + const visitedContainerIds = new Set() + + const findAncestorContainers = (targetId: string) => { + if (visitedContainerIds.has(targetId)) return + visitedContainerIds.add(targetId) + + // Check if targetId is directly inside any loop + for (const [loopId, loop] of Object.entries(loops)) { + if (loop.nodes.includes(targetId) && !ancestorLoopIds.has(loopId)) { + ancestorLoopIds.add(loopId) + const loopBlock = blocks[loopId] + if (loopBlock) { + const loopType = loop.loopType || 'for' + const loopBlockName = loopBlock.name || loopBlock.type + const normalizedLoopName = normalizeName(loopBlockName) + const contextualTags: string[] = [`${normalizedLoopName}.index`] + if (loopType === 'forEach') { + contextualTags.push(`${normalizedLoopName}.currentItem`) + contextualTags.push(`${normalizedLoopName}.items`) + } + loopBlockGroups.push({ + blockName: loopBlockName, + blockId: loopId, + blockType: 'loop', + tags: contextualTags, + distance: 0, + isContextual: true, + }) + } + findAncestorContainers(loopId) + } + } + // Also walk through containing parallels so we find loops that contain + // the parallel (e.g. block inside parallel inside loop) + for (const [parallelId, parallel] of Object.entries(parallels || {})) { + if (parallel.nodes.includes(targetId)) { + findAncestorContainers(parallelId) + } + } + } const isLoopBlock = blocks[blockId]?.type === 'loop' - const currentLoop = isLoopBlock ? loops[blockId] : null - - const containingLoop = Object.entries(loops).find(([_, loop]) => loop.nodes.includes(blockId)) - - let containingLoopBlockId: string | null = null - - if (currentLoop && isLoopBlock) { - containingLoopBlockId = blockId - const loopType = currentLoop.loopType || 'for' - + if (isLoopBlock && loops[blockId]) { + const loop = loops[blockId] + ancestorLoopIds.add(blockId) const loopBlock = blocks[blockId] if (loopBlock) { + const loopType = loop.loopType || 'for' const loopBlockName = loopBlock.name || loopBlock.type const normalizedLoopName = normalizeName(loopBlockName) const contextualTags: string[] = [`${normalizedLoopName}.index`] @@ -1189,71 +1224,65 @@ export const TagDropdown: React.FC = ({ contextualTags.push(`${normalizedLoopName}.currentItem`) contextualTags.push(`${normalizedLoopName}.items`) } - - loopBlockGroup = { + loopBlockGroups.push({ blockName: loopBlockName, blockId: blockId, blockType: 'loop', tags: contextualTags, distance: 0, isContextual: true, - } + }) } - } else if (containingLoop) { - const [loopId, loop] = containingLoop - containingLoopBlockId = loopId - const loopType = loop.loopType || 'for' + findAncestorContainers(blockId) + } else { + findAncestorContainers(blockId) + } - const containingLoopBlock = blocks[loopId] - if (containingLoopBlock) { - const loopBlockName = containingLoopBlock.name || containingLoopBlock.type - const normalizedLoopName = normalizeName(loopBlockName) - const contextualTags: string[] = [`${normalizedLoopName}.index`] - if (loopType === 'forEach') { - contextualTags.push(`${normalizedLoopName}.currentItem`) - contextualTags.push(`${normalizedLoopName}.items`) - } + const parallelBlockGroups: BlockTagGroup[] = [] + const ancestorParallelIds = new Set() + const visitedParallelTargets = new Set() - loopBlockGroup = { - blockName: loopBlockName, - blockId: loopId, - blockType: 'loop', - tags: contextualTags, - distance: 0, - isContextual: true, + const findAncestorParallels = (targetId: string) => { + if (visitedParallelTargets.has(targetId)) return + visitedParallelTargets.add(targetId) + + for (const [parallelId, parallel] of Object.entries(parallels || {})) { + if (parallel.nodes.includes(targetId) && !ancestorParallelIds.has(parallelId)) { + ancestorParallelIds.add(parallelId) + const parallelBlock = blocks[parallelId] + if (parallelBlock) { + const parallelType = parallel.parallelType || 'count' + const parallelBlockName = parallelBlock.name || parallelBlock.type + const normalizedParallelName = normalizeName(parallelBlockName) + const contextualTags: string[] = [`${normalizedParallelName}.index`] + if (parallelType === 'collection') { + contextualTags.push(`${normalizedParallelName}.currentItem`) + contextualTags.push(`${normalizedParallelName}.items`) + } + parallelBlockGroups.push({ + blockName: parallelBlockName, + blockId: parallelId, + blockType: 'parallel', + tags: contextualTags, + distance: 0, + isContextual: true, + }) + } + // Walk up through containing loops and parallels + for (const [loopId, loop] of Object.entries(loops)) { + if (loop.nodes.includes(parallelId)) { + findAncestorParallels(loopId) + } + } + findAncestorParallels(parallelId) } } } - let parallelBlockGroup: BlockTagGroup | null = null - const containingParallel = Object.entries(parallels || {}).find(([_, parallel]) => - parallel.nodes.includes(blockId) - ) - let containingParallelBlockId: string | null = null - if (containingParallel) { - const [parallelId, parallel] = containingParallel - containingParallelBlockId = parallelId - const parallelType = parallel.parallelType || 'count' - - const containingParallelBlock = blocks[parallelId] - if (containingParallelBlock) { - const parallelBlockName = containingParallelBlock.name || containingParallelBlock.type - const normalizedParallelName = normalizeName(parallelBlockName) - const contextualTags: string[] = [`${normalizedParallelName}.index`] - if (parallelType === 'collection') { - contextualTags.push(`${normalizedParallelName}.currentItem`) - contextualTags.push(`${normalizedParallelName}.items`) - } - - parallelBlockGroup = { - blockName: parallelBlockName, - blockId: parallelId, - blockType: 'parallel', - tags: contextualTags, - distance: 0, - isContextual: true, - } - } + findAncestorParallels(blockId) + // Also check through ancestor loops (a block in a loop that's in a parallel) + for (const loopId of ancestorLoopIds) { + findAncestorParallels(loopId) } const blockTagGroups: BlockTagGroup[] = [] @@ -1275,8 +1304,8 @@ export const TagDropdown: React.FC = ({ if (!blockConfig) { if (accessibleBlock.type === 'loop' || accessibleBlock.type === 'parallel') { if ( - accessibleBlockId === containingLoopBlockId || - accessibleBlockId === containingParallelBlockId + ancestorLoopIds.has(accessibleBlockId) || + ancestorParallelIds.has(accessibleBlockId) ) { continue } @@ -1366,12 +1395,8 @@ export const TagDropdown: React.FC = ({ } const finalBlockTagGroups: BlockTagGroup[] = [] - if (loopBlockGroup) { - finalBlockTagGroups.push(loopBlockGroup) - } - if (parallelBlockGroup) { - finalBlockTagGroups.push(parallelBlockGroup) - } + finalBlockTagGroups.push(...loopBlockGroups) + finalBlockTagGroups.push(...parallelBlockGroups) blockTagGroups.sort((a, b) => a.distance - b.distance) finalBlockTagGroups.push(...blockTagGroups) @@ -1570,21 +1595,6 @@ export const TagDropdown: React.FC = ({ if (variableObj) { processedTag = tag } - } else if ( - blockGroup?.isContextual && - (blockGroup.blockType === 'loop' || blockGroup.blockType === 'parallel') - ) { - const tagParts = tag.split('.') - if (tagParts.length === 1) { - processedTag = blockGroup.blockType - } else { - const lastPart = tagParts[tagParts.length - 1] - if (['index', 'currentItem', 'items'].includes(lastPart)) { - processedTag = `${blockGroup.blockType}.${lastPart}` - } else { - processedTag = tag - } - } } let newValue: string diff --git a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/toolbar/toolbar.tsx b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/toolbar/toolbar.tsx index 365379854a..d215f5576c 100644 --- a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/toolbar/toolbar.tsx +++ b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/toolbar/toolbar.tsx @@ -67,9 +67,6 @@ const ToolbarItem = memo(function ToolbarItem({ const handleDragStart = useCallback( (e: React.DragEvent) => { - if (!isTrigger && (item.type === 'loop' || item.type === 'parallel')) { - document.body.classList.add('sim-drag-subflow') - } const iconElement = e.currentTarget.querySelector('.toolbar-item-icon') onDragStart(e, item.type, isTriggerCapable, { name: item.name, @@ -80,12 +77,6 @@ const ToolbarItem = memo(function ToolbarItem({ [item.type, item.name, item.bgColor, isTriggerCapable, onDragStart, isTrigger] ) - const handleDragEnd = useCallback(() => { - if (!isTrigger) { - document.body.classList.remove('sim-drag-subflow') - } - }, [isTrigger]) - const handleClick = useCallback(() => { onClick(item.type, isTriggerCapable) }, [item.type, isTriggerCapable, onClick]) @@ -114,7 +105,6 @@ const ToolbarItem = memo(function ToolbarItem({ tabIndex={-1} draggable onDragStart={handleDragStart} - onDragEnd={handleDragEnd} onClick={handleClick} onContextMenu={handleContextMenu} className={clsx( diff --git a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/subflows/subflow-node.tsx b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/subflows/subflow-node.tsx index 6205818cdd..5c1c0db84c 100644 --- a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/subflows/subflow-node.tsx +++ b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/subflows/subflow-node.tsx @@ -8,6 +8,7 @@ import { type DiffStatus, hasDiffStatus } from '@/lib/workflows/diff/types' import { useUserPermissionsContext } from '@/app/workspace/[workspaceId]/providers/workspace-permissions-provider' import { ActionBar } from '@/app/workspace/[workspaceId]/w/[workflowId]/components/action-bar/action-bar' import { useCurrentWorkflow } from '@/app/workspace/[workspaceId]/w/[workflowId]/hooks' +import { useLastRunPath } from '@/stores/execution' import { usePanelEditorStore } from '@/stores/panel' /** @@ -23,6 +24,8 @@ export interface SubflowNodeData { isPreviewSelected?: boolean kind: 'loop' | 'parallel' name?: string + /** Execution status passed by preview/snapshot views */ + executionStatus?: 'success' | 'error' | 'not-executed' } /** @@ -56,6 +59,15 @@ export const SubflowNodeComponent = memo(({ data, id, selected }: NodeProps
setCurrentBlockId(id)} className={cn( - 'workflow-drag-handle relative cursor-grab select-none rounded-[8px] border border-[var(--border-1)] [&:active]:cursor-grabbing', - 'transition-block-bg transition-ring', - 'z-[20]' + 'relative select-none rounded-[8px] border border-[var(--border-1)]', + 'transition-block-bg' )} style={{ width: data.width || 500, height: data.height || 300, position: 'relative', overflow: 'visible', - pointerEvents: isPreview ? 'none' : 'all', + pointerEvents: 'none', + ...(outlineColor && { + outline: `1.75px solid ${outlineColor}`, + outlineOffset: '-1px', + }), }} data-node-id={id} data-type='subflowNode' @@ -142,11 +178,13 @@ export const SubflowNodeComponent = memo(({ data, id, selected }: NodeProps )} - {/* Header Section */} + {/* Header Section — only interactive area for dragging */}
setCurrentBlockId(id)} className={cn( - 'flex items-center justify-between rounded-t-[8px] border-[var(--border)] border-b bg-[var(--surface-2)] py-[8px] pr-[12px] pl-[8px]' + 'workflow-drag-handle flex cursor-grab items-center justify-between rounded-t-[8px] border-[var(--border)] border-b bg-[var(--surface-2)] py-[8px] pr-[12px] pl-[8px] [&:active]:cursor-grabbing' )} + style={{ pointerEvents: 'auto' }} >
{/* Subflow Start */} @@ -233,12 +271,6 @@ export const SubflowNodeComponent = memo(({ data, id, selected }: NodeProps - - {hasRing && ( -
- )}
) diff --git a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/terminal/terminal.tsx b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/terminal/terminal.tsx index 204ca166c8..895ecc912a 100644 --- a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/terminal/terminal.tsx +++ b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/terminal/terminal.tsx @@ -60,6 +60,7 @@ import { openCopilotWithMessage } from '@/stores/notifications/utils' import type { ConsoleEntry } from '@/stores/terminal' import { useTerminalConsoleStore, useTerminalStore } from '@/stores/terminal' import { useWorkflowRegistry } from '@/stores/workflows/registry/store' +import { useWorkflowStore } from '@/stores/workflows/workflow/store' /** * Terminal height configuration constants @@ -68,20 +69,21 @@ const MIN_HEIGHT = TERMINAL_HEIGHT.MIN const DEFAULT_EXPANDED_HEIGHT = TERMINAL_HEIGHT.DEFAULT const MIN_OUTPUT_PANEL_WIDTH_PX = OUTPUT_PANEL_WIDTH.MIN -/** Returns true if any node in the subtree has an error */ -function hasErrorInTree(nodes: EntryNode[]): boolean { - return nodes.some((n) => Boolean(n.entry.error) || hasErrorInTree(n.children)) +const MAX_TREE_DEPTH = 50 + +function hasMatchInTree( + nodes: EntryNode[], + predicate: (e: ConsoleEntry) => boolean, + depth = 0 +): boolean { + if (depth >= MAX_TREE_DEPTH) return false + return nodes.some((n) => predicate(n.entry) || hasMatchInTree(n.children, predicate, depth + 1)) } -/** Returns true if any node in the subtree is currently running */ -function hasRunningInTree(nodes: EntryNode[]): boolean { - return nodes.some((n) => Boolean(n.entry.isRunning) || hasRunningInTree(n.children)) -} - -/** Returns true if any node in the subtree was canceled */ -function hasCanceledInTree(nodes: EntryNode[]): boolean { - return nodes.some((n) => Boolean(n.entry.isCanceled) || hasCanceledInTree(n.children)) -} +const hasErrorInTree = (nodes: EntryNode[]) => hasMatchInTree(nodes, (e) => Boolean(e.error)) +const hasRunningInTree = (nodes: EntryNode[]) => hasMatchInTree(nodes, (e) => Boolean(e.isRunning)) +const hasCanceledInTree = (nodes: EntryNode[]) => + hasMatchInTree(nodes, (e) => Boolean(e.isCanceled)) /** * Block row component for displaying actual block entries @@ -263,28 +265,21 @@ const SubflowNodeRow = memo(function SubflowNodeRow({ }) { const { entry, children } = node const BlockIcon = getBlockIcon(entry.blockType) - const hasError = - Boolean(entry.error) || - children.some((c) => c.entry.error || c.children.some((gc) => gc.entry.error)) + const hasError = Boolean(entry.error) || hasErrorInTree(children) const bgColor = getBlockColor(entry.blockType) const nodeId = entry.id const isExpanded = expandedNodes.has(nodeId) const hasChildren = children.length > 0 - // Check if any nested block is running or canceled - const hasRunningDescendant = children.some( - (c) => c.entry.isRunning || c.children.some((gc) => gc.entry.isRunning) - ) - const hasCanceledDescendant = - children.some((c) => c.entry.isCanceled || c.children.some((gc) => gc.entry.isCanceled)) && - !hasRunningDescendant + // Check if any nested block is running or canceled (recursive for arbitrary nesting depth) + const hasRunningDescendant = hasRunningInTree(children) + const hasCanceledDescendant = hasCanceledInTree(children) && !hasRunningDescendant - const displayName = - entry.blockType === 'loop' - ? 'Loop' - : entry.blockType === 'parallel' - ? 'Parallel' - : entry.blockName + const containerId = entry.iterationContainerId + const storeBlockName = useWorkflowStore((state) => + containerId ? state.blocks[containerId]?.name : undefined + ) + const displayName = storeBlockName || entry.blockName return (
diff --git a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/terminal/utils.test.ts b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/terminal/utils.test.ts new file mode 100644 index 0000000000..c27f792aa3 --- /dev/null +++ b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/terminal/utils.test.ts @@ -0,0 +1,478 @@ +/** + * @vitest-environment node + */ +import { describe, expect, it, vi } from 'vitest' + +vi.mock('@/blocks', () => ({ + getBlock: vi.fn().mockReturnValue(null), +})) + +vi.mock('@/executor/constants', () => ({ + isWorkflowBlockType: vi.fn((blockType: string | undefined) => { + return blockType === 'workflow' || blockType === 'workflow_input' + }), +})) + +vi.mock('@/stores/constants', () => ({ + TERMINAL_BLOCK_COLUMN_WIDTH: { MIN: 120, DEFAULT: 200, MAX: 400 }, +})) + +import type { ConsoleEntry } from '@/stores/terminal' +import { buildEntryTree, type EntryNode, groupEntriesByExecution } from './utils' + +let entryCounter = 0 + +function makeEntry(overrides: Partial): ConsoleEntry { + return { + id: overrides.id ?? `entry-${++entryCounter}`, + timestamp: overrides.timestamp ?? '2025-01-01T00:00:00Z', + workflowId: overrides.workflowId ?? 'wf-1', + blockId: overrides.blockId ?? 'block-1', + blockName: overrides.blockName ?? 'Block', + blockType: overrides.blockType ?? 'function', + executionId: overrides.executionId ?? 'exec-1', + startedAt: overrides.startedAt ?? '2025-01-01T00:00:00Z', + executionOrder: overrides.executionOrder ?? 0, + ...overrides, + } as ConsoleEntry +} + +/** Collect all nodes from a tree depth-first */ +function collectAllNodes(nodes: EntryNode[]): EntryNode[] { + const result: EntryNode[] = [] + for (const node of nodes) { + result.push(node) + result.push(...collectAllNodes(node.children)) + } + return result +} + +/** + * Creates entries for a parallel-in-loop scenario. + * All Function 1 entries are nestedIterationEntries (have parentIterations). + * No topLevelIterationEntries exist (sentinels don't emit SSE events). + */ +function makeParallelInLoopEntries( + loopIterations: number, + parallelBranches: number +): ConsoleEntry[] { + const entries: ConsoleEntry[] = [] + let order = 1 + for (let loopIter = 0; loopIter < loopIterations; loopIter++) { + for (let branch = 0; branch < parallelBranches; branch++) { + entries.push( + makeEntry({ + blockId: 'function-1', + blockName: 'Function 1', + executionOrder: order++, + startedAt: new Date(Date.UTC(2025, 0, 1, 0, 0, loopIter * 10 + branch)).toISOString(), + endedAt: new Date(Date.UTC(2025, 0, 1, 0, 0, loopIter * 10 + branch + 1)).toISOString(), + durationMs: 50, + iterationType: 'parallel', + iterationCurrent: branch, + iterationTotal: parallelBranches, + iterationContainerId: 'parallel-1', + parentIterations: [ + { + iterationType: 'loop', + iterationCurrent: loopIter, + iterationTotal: loopIterations, + iterationContainerId: 'loop-1', + }, + ], + }) + ) + } + } + return entries +} + +describe('buildEntryTree', () => { + describe('simple loop (no nesting)', () => { + it('groups entries by loop iteration', () => { + const entries: ConsoleEntry[] = [] + for (let iter = 0; iter < 3; iter++) { + entries.push( + makeEntry({ + blockId: 'function-1', + blockName: 'Function 1', + executionOrder: iter + 1, + iterationType: 'loop', + iterationCurrent: iter, + iterationTotal: 3, + iterationContainerId: 'loop-1', + }) + ) + } + + const tree = buildEntryTree(entries) + + const subflows = tree.filter((n) => n.nodeType === 'subflow') + expect(subflows).toHaveLength(1) + expect(subflows[0].entry.blockType).toBe('loop') + expect(subflows[0].children).toHaveLength(3) + + for (let i = 0; i < 3; i++) { + expect(subflows[0].children[i].iterationInfo?.current).toBe(i) + expect(subflows[0].children[i].children).toHaveLength(1) + expect(subflows[0].children[i].children[0].entry.blockId).toBe('function-1') + } + }) + }) + + describe('simple parallel (no nesting)', () => { + it('groups entries by parallel branch', () => { + const entries: ConsoleEntry[] = [] + for (let branch = 0; branch < 4; branch++) { + entries.push( + makeEntry({ + blockId: 'function-1', + blockName: 'Function 1', + executionOrder: branch + 1, + iterationType: 'parallel', + iterationCurrent: branch, + iterationTotal: 4, + iterationContainerId: 'parallel-1', + }) + ) + } + + const tree = buildEntryTree(entries) + + const subflows = tree.filter((n) => n.nodeType === 'subflow') + expect(subflows).toHaveLength(1) + expect(subflows[0].entry.blockType).toBe('parallel') + expect(subflows[0].children).toHaveLength(4) + }) + }) + + describe('parallel-in-loop', () => { + it('creates all loop iterations (5 loop × 5 parallel)', () => { + const entries = makeParallelInLoopEntries(5, 5) + expect(entries).toHaveLength(25) + + const tree = buildEntryTree(entries) + + // Top level: 1 subflow (Loop) + const subflows = tree.filter((n) => n.nodeType === 'subflow') + expect(subflows).toHaveLength(1) + expect(subflows[0].entry.blockType).toBe('loop') + + // Loop has 5 iteration children + const loopIterations = subflows[0].children + expect(loopIterations).toHaveLength(5) + + for (let loopIter = 0; loopIter < 5; loopIter++) { + const iterNode = loopIterations[loopIter] + expect(iterNode.nodeType).toBe('iteration') + expect(iterNode.iterationInfo?.current).toBe(loopIter) + expect(iterNode.iterationInfo?.total).toBe(5) + + // Each loop iteration has 1 nested subflow (Parallel) + const parallelSubflows = iterNode.children.filter((n) => n.nodeType === 'subflow') + expect(parallelSubflows).toHaveLength(1) + expect(parallelSubflows[0].entry.blockType).toBe('parallel') + + // Each parallel has 5 branch iterations + const branches = parallelSubflows[0].children + expect(branches).toHaveLength(5) + for (let branch = 0; branch < 5; branch++) { + expect(branches[branch].iterationInfo?.current).toBe(branch) + expect(branches[branch].children).toHaveLength(1) + expect(branches[branch].children[0].entry.blockId).toBe('function-1') + } + } + }) + + it('preserves all block entries in the tree (no silently dropped entries)', () => { + const entries = makeParallelInLoopEntries(5, 5) + const tree = buildEntryTree(entries) + + const allNodes = collectAllNodes(tree) + const blocks = allNodes.filter( + (n) => n.nodeType === 'block' && n.entry.blockId === 'function-1' + ) + expect(blocks).toHaveLength(25) + }) + + it('works with a regular block alongside', () => { + const entries = [ + makeEntry({ + blockId: 'start-1', + blockName: 'Start', + blockType: 'starter', + executionOrder: 0, + }), + ...makeParallelInLoopEntries(3, 2), + ] + + const tree = buildEntryTree(entries) + + const regularBlocks = tree.filter((n) => n.nodeType === 'block') + const subflows = tree.filter((n) => n.nodeType === 'subflow') + expect(regularBlocks).toHaveLength(1) + expect(regularBlocks[0].entry.blockId).toBe('start-1') + expect(subflows).toHaveLength(1) + expect(subflows[0].children).toHaveLength(3) + }) + + it('works when some iterations also have topLevelIterationEntries', () => { + const entries: ConsoleEntry[] = [ + // Real top-level entry for loop iteration 0 (from a container event) + makeEntry({ + blockId: 'parallel-container', + blockName: 'Parallel', + blockType: 'parallel', + executionOrder: 100, + iterationType: 'loop', + iterationCurrent: 0, + iterationTotal: 3, + iterationContainerId: 'loop-1', + }), + ...makeParallelInLoopEntries(3, 2), + ] + + const tree = buildEntryTree(entries) + const subflows = tree.filter((n) => n.nodeType === 'subflow') + expect(subflows).toHaveLength(1) + + // All 3 loop iterations must exist + expect(subflows[0].children).toHaveLength(3) + + // All 6 Function 1 blocks should appear somewhere in the tree + const allNodes = collectAllNodes(tree) + const fnBlocks = allNodes.filter( + (n) => n.nodeType === 'block' && n.entry.blockId === 'function-1' + ) + expect(fnBlocks).toHaveLength(6) + }) + + it('handles 2 loop × 3 parallel', () => { + const entries = makeParallelInLoopEntries(2, 3) + const tree = buildEntryTree(entries) + + const subflows = tree.filter((n) => n.nodeType === 'subflow') + expect(subflows).toHaveLength(1) + expect(subflows[0].children).toHaveLength(2) + + const allNodes = collectAllNodes(tree) + const blocks = allNodes.filter( + (n) => n.nodeType === 'block' && n.entry.blockId === 'function-1' + ) + expect(blocks).toHaveLength(6) + }) + }) + + describe('loop-in-parallel', () => { + it('creates all parallel branches with nested loop iterations', () => { + const entries: ConsoleEntry[] = [] + let order = 1 + for (let branch = 0; branch < 3; branch++) { + for (let loopIter = 0; loopIter < 2; loopIter++) { + entries.push( + makeEntry({ + blockId: 'function-1', + blockName: 'Function 1', + executionOrder: order++, + iterationType: 'loop', + iterationCurrent: loopIter, + iterationTotal: 2, + iterationContainerId: 'loop-1', + parentIterations: [ + { + iterationType: 'parallel', + iterationCurrent: branch, + iterationTotal: 3, + iterationContainerId: 'parallel-1', + }, + ], + }) + ) + } + } + + const tree = buildEntryTree(entries) + + const subflows = tree.filter((n) => n.nodeType === 'subflow') + expect(subflows).toHaveLength(1) + expect(subflows[0].entry.blockType).toBe('parallel') + + // 3 parallel branches + const branches = subflows[0].children + expect(branches).toHaveLength(3) + + for (let branch = 0; branch < 3; branch++) { + const branchNode = branches[branch] + expect(branchNode.iterationInfo?.current).toBe(branch) + + // Each branch has a nested loop subflow + const nestedSubflows = branchNode.children.filter((n) => n.nodeType === 'subflow') + expect(nestedSubflows).toHaveLength(1) + expect(nestedSubflows[0].entry.blockType).toBe('loop') + + // Each loop has 2 iterations + expect(nestedSubflows[0].children).toHaveLength(2) + } + }) + }) + + describe('loop-in-loop', () => { + it('creates outer and inner loop iterations', () => { + const entries: ConsoleEntry[] = [] + let order = 1 + for (let outer = 0; outer < 2; outer++) { + for (let inner = 0; inner < 3; inner++) { + entries.push( + makeEntry({ + blockId: 'function-1', + blockName: 'Function 1', + executionOrder: order++, + iterationType: 'loop', + iterationCurrent: inner, + iterationTotal: 3, + iterationContainerId: 'inner-loop', + parentIterations: [ + { + iterationType: 'loop', + iterationCurrent: outer, + iterationTotal: 2, + iterationContainerId: 'outer-loop', + }, + ], + }) + ) + } + } + + const tree = buildEntryTree(entries) + + const subflows = tree.filter((n) => n.nodeType === 'subflow') + expect(subflows).toHaveLength(1) + expect(subflows[0].entry.blockType).toBe('loop') + + // Outer loop: 2 iterations + expect(subflows[0].children).toHaveLength(2) + + for (let outer = 0; outer < 2; outer++) { + const outerIter = subflows[0].children[outer] + expect(outerIter.iterationInfo?.current).toBe(outer) + + // Each outer iteration has an inner loop + const innerSubflows = outerIter.children.filter((n) => n.nodeType === 'subflow') + expect(innerSubflows).toHaveLength(1) + expect(innerSubflows[0].children).toHaveLength(3) + } + + // All 6 blocks present + const allNodes = collectAllNodes(tree) + const blocks = allNodes.filter((n) => n.nodeType === 'block') + expect(blocks).toHaveLength(6) + }) + }) + + describe('parallel-in-parallel', () => { + it('creates outer and inner parallel branches', () => { + const entries: ConsoleEntry[] = [] + let order = 1 + for (let outer = 0; outer < 2; outer++) { + for (let inner = 0; inner < 3; inner++) { + entries.push( + makeEntry({ + blockId: 'function-1', + blockName: 'Function 1', + executionOrder: order++, + iterationType: 'parallel', + iterationCurrent: inner, + iterationTotal: 3, + iterationContainerId: 'inner-parallel', + parentIterations: [ + { + iterationType: 'parallel', + iterationCurrent: outer, + iterationTotal: 2, + iterationContainerId: 'outer-parallel', + }, + ], + }) + ) + } + } + + const tree = buildEntryTree(entries) + + const subflows = tree.filter((n) => n.nodeType === 'subflow') + expect(subflows).toHaveLength(1) + expect(subflows[0].entry.blockType).toBe('parallel') + + // 2 outer branches + expect(subflows[0].children).toHaveLength(2) + + for (let outer = 0; outer < 2; outer++) { + const outerBranch = subflows[0].children[outer] + const innerSubflows = outerBranch.children.filter((n) => n.nodeType === 'subflow') + expect(innerSubflows).toHaveLength(1) + expect(innerSubflows[0].children).toHaveLength(3) + } + + const allNodes = collectAllNodes(tree) + const blocks = allNodes.filter((n) => n.nodeType === 'block') + expect(blocks).toHaveLength(6) + }) + }) +}) + +describe('groupEntriesByExecution', () => { + it('builds tree for parallel-in-loop via groupEntriesByExecution', () => { + const entries = makeParallelInLoopEntries(3, 2) + + const groups = groupEntriesByExecution(entries) + expect(groups).toHaveLength(1) + + const entryTree = groups[0].entryTree + const subflows = entryTree.filter((n) => n.nodeType === 'subflow') + expect(subflows).toHaveLength(1) + expect(subflows[0].children).toHaveLength(3) + }) + + it('handles workflow child entries alongside iteration entries', () => { + const entries: ConsoleEntry[] = [ + makeEntry({ + id: 'start-entry', + blockId: 'start', + blockName: 'Start', + blockType: 'start_trigger', + executionOrder: 0, + }), + makeEntry({ + id: 'workflow-block', + blockId: 'wf-block-1', + blockName: 'My Sub-Workflow', + blockType: 'workflow', + executionOrder: 1, + }), + makeEntry({ + id: 'child-block-1', + blockId: 'child-func', + blockName: 'Child Function', + blockType: 'function', + executionOrder: 2, + childWorkflowBlockId: 'wf-block-1', + childWorkflowName: 'Child Workflow', + childWorkflowInstanceId: 'instance-1', + }), + ] + + const groups = groupEntriesByExecution(entries) + expect(groups).toHaveLength(1) + + const tree = groups[0].entryTree + expect(tree.length).toBeGreaterThanOrEqual(2) + + const startNode = tree.find((n) => n.entry.blockType === 'start_trigger') + expect(startNode).toBeDefined() + + // Child entry should be nested under workflow block, not at top level + const topLevelChild = tree.find((n) => n.entry.blockId === 'child-func') + expect(topLevelChild).toBeUndefined() + }) +}) diff --git a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/terminal/utils.ts b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/terminal/utils.ts index a31bf2cc1d..c4c6980b98 100644 --- a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/terminal/utils.ts +++ b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/terminal/utils.ts @@ -18,10 +18,9 @@ import type { ConsoleEntry } from '@/stores/terminal' const SUBFLOW_COLORS = { loop: '#2FB3FF', parallel: '#FEE12B', + workflow: '#8b5cf6', } as const -const WORKFLOW_COLOR = '#8b5cf6' - /** * Special block type colors for errors and system messages */ @@ -86,7 +85,7 @@ export function getBlockColor(blockType: string): string { return SUBFLOW_COLORS.parallel } if (blockType === 'workflow') { - return WORKFLOW_COLOR + return SUBFLOW_COLORS.workflow } // Special block types for errors and system messages if (blockType === 'error') { @@ -126,14 +125,6 @@ export function isEventFromEditableElement(e: KeyboardEvent): boolean { return false } -/** - * Checks if a block type is a subflow (loop or parallel) - */ -export function isSubflowBlockType(blockType: string): boolean { - const lower = blockType?.toLowerCase() || '' - return lower === 'loop' || lower === 'parallel' -} - /** * Node type for the tree structure */ @@ -221,27 +212,26 @@ function collectWorkflowDescendants( * that executed within each iteration. * Sorts by start time to ensure chronological order. */ -function buildEntryTree(entries: ConsoleEntry[]): EntryNode[] { - // Separate entries into three buckets: - // 1. Iteration entries (loop/parallel children) - // 2. Workflow child entries (blocks inside a child workflow) - // 3. Regular blocks +export function buildEntryTree(entries: ConsoleEntry[], idPrefix = ''): EntryNode[] { const regularBlocks: ConsoleEntry[] = [] - const iterationEntries: ConsoleEntry[] = [] + const topLevelIterationEntries: ConsoleEntry[] = [] + const nestedIterationEntries: ConsoleEntry[] = [] const workflowChildEntries: ConsoleEntry[] = [] for (const entry of entries) { if (entry.childWorkflowBlockId) { - // Child workflow entries take priority over iteration classification workflowChildEntries.push(entry) } else if (entry.iterationType && entry.iterationCurrent !== undefined) { - iterationEntries.push(entry) + if (entry.parentIterations && entry.parentIterations.length > 0) { + nestedIterationEntries.push(entry) + } else { + topLevelIterationEntries.push(entry) + } } else { regularBlocks.push(entry) } } - // Group workflow child entries by the parent workflow block ID const workflowChildGroups = new Map() for (const entry of workflowChildEntries) { const parentId = entry.childWorkflowBlockId! @@ -253,9 +243,8 @@ function buildEntryTree(entries: ConsoleEntry[]): EntryNode[] { } } - // Group iteration entries by (iterationType, iterationContainerId, iterationCurrent) const iterationGroupsMap = new Map() - for (const entry of iterationEntries) { + for (const entry of topLevelIterationEntries) { const iterationContainerId = entry.iterationContainerId || 'unknown' const key = `${entry.iterationType}-${iterationContainerId}-${entry.iterationCurrent}` let group = iterationGroupsMap.get(key) @@ -272,11 +261,9 @@ function buildEntryTree(entries: ConsoleEntry[]): EntryNode[] { } iterationGroupsMap.set(key, group) } else { - // Update start time to earliest if (entryStartMs < group.startTimeMs) { group.startTimeMs = entryStartMs } - // Update total if available if (entry.iterationTotal !== undefined) { group.iterationTotal = entry.iterationTotal } @@ -284,12 +271,10 @@ function buildEntryTree(entries: ConsoleEntry[]): EntryNode[] { group.blocks.push(entry) } - // Sort blocks within each iteration by executionOrder ascending (oldest first, top-down) for (const group of iterationGroupsMap.values()) { group.blocks.sort((a, b) => a.executionOrder - b.executionOrder) } - // Group iterations by (iterationType, iterationContainerId) to create subflow parents const subflowGroups = new Map< string, { iterationType: string; iterationContainerId: string; groups: IterationGroup[] } @@ -308,112 +293,218 @@ function buildEntryTree(entries: ConsoleEntry[]): EntryNode[] { subflowGroup.groups.push(group) } - // Sort iterations within each subflow by iteration number for (const subflowGroup of subflowGroups.values()) { subflowGroup.groups.sort((a, b) => a.iterationCurrent - b.iterationCurrent) } - // Build subflow nodes with iteration children + // Create synthetic parent subflow groups for orphaned nested iteration entries. + // Nested subflow containers (e.g., inner parallel inside outer parallel) may not + // have store entries if no block:started event was emitted for them. Without a + // parent subflow group, their child entries would be silently dropped from the tree. + // Check at the iteration level (not container level) so that existing iterations + // from topLevelIterationEntries don't block synthetic creation for other iterations + // of the same container (e.g., loop iterations 1-4 when iteration 0 already exists). + const syntheticIterations = new Map() + for (const entry of nestedIterationEntries) { + const parent = entry.parentIterations?.[0] + if (!parent?.iterationContainerId) { + continue + } + + // Only skip if this specific iteration already has a group from topLevelIterationEntries + const iterKey = `${parent.iterationType}-${parent.iterationContainerId}-${parent.iterationCurrent}` + if (iterationGroupsMap.has(iterKey)) { + continue + } + + const entryMs = new Date(entry.startedAt || entry.timestamp).getTime() + if (!syntheticIterations.has(iterKey)) { + syntheticIterations.set(iterKey, { + iterationType: parent.iterationType!, + iterationContainerId: parent.iterationContainerId!, + iterationCurrent: parent.iterationCurrent!, + iterationTotal: parent.iterationTotal, + blocks: [], + startTimeMs: entryMs, + }) + } else { + const existing = syntheticIterations.get(iterKey)! + if (entryMs < existing.startTimeMs) { + existing.startTimeMs = entryMs + } + } + } + + const syntheticSubflows = new Map< + string, + { iterationType: string; iterationContainerId: string; groups: IterationGroup[] } + >() + for (const iterGroup of syntheticIterations.values()) { + const subflowKey = `${iterGroup.iterationType}-${iterGroup.iterationContainerId}` + let subflow = syntheticSubflows.get(subflowKey) + if (!subflow) { + subflow = { + iterationType: iterGroup.iterationType, + iterationContainerId: iterGroup.iterationContainerId, + groups: [], + } + syntheticSubflows.set(subflowKey, subflow) + } + subflow.groups.push(iterGroup) + } + + for (const subflow of syntheticSubflows.values()) { + const key = `${subflow.iterationType}-${subflow.iterationContainerId}` + const existing = subflowGroups.get(key) + if (existing) { + // Merge synthetic iteration groups into the existing subflow group + existing.groups.push(...subflow.groups) + existing.groups.sort((a, b) => a.iterationCurrent - b.iterationCurrent) + } else { + subflow.groups.sort((a, b) => a.iterationCurrent - b.iterationCurrent) + subflowGroups.set(key, subflow) + } + } + const subflowNodes: EntryNode[] = [] for (const subflowGroup of subflowGroups.values()) { const { iterationType, iterationContainerId, groups: iterationGroups } = subflowGroup - // Calculate subflow timing from all its iterations - const firstIteration = iterationGroups[0] - const allBlocks = iterationGroups.flatMap((g) => g.blocks) - const subflowStartMs = Math.min( - ...allBlocks.map((b) => new Date(b.startedAt || b.timestamp).getTime()) - ) + + const nestedForThisSubflow = nestedIterationEntries.filter((e) => { + const parent = e.parentIterations?.[0] + return parent && parent.iterationContainerId === iterationContainerId + }) + + const allDirectBlocks = iterationGroups.flatMap((g) => g.blocks) + const allRelevantBlocks = [...allDirectBlocks, ...nestedForThisSubflow] + if (allRelevantBlocks.length === 0) continue + + const timestamps = allRelevantBlocks.map((b) => new Date(b.startedAt || b.timestamp).getTime()) + const subflowStartMs = Math.min(...timestamps) const subflowEndMs = Math.max( - ...allBlocks.map((b) => new Date(b.endedAt || b.timestamp).getTime()) + ...allRelevantBlocks.map((b) => new Date(b.endedAt || b.timestamp).getTime()) ) - const totalDuration = allBlocks.reduce((sum, b) => sum + (b.durationMs || 0), 0) - // Parallel branches run concurrently — use wall-clock time. Loop iterations run serially — use sum. + const totalDuration = allRelevantBlocks.reduce((sum, b) => sum + (b.durationMs || 0), 0) const subflowDuration = iterationType === 'parallel' ? subflowEndMs - subflowStartMs : totalDuration - // Create synthetic subflow parent entry - // Use the minimum executionOrder from all child blocks for proper ordering - const subflowExecutionOrder = Math.min(...allBlocks.map((b) => b.executionOrder)) + const subflowExecutionOrder = Math.min(...allRelevantBlocks.map((b) => b.executionOrder)) + const metadataSource = allRelevantBlocks[0] const syntheticSubflow: ConsoleEntry = { - id: `subflow-${iterationType}-${iterationContainerId}-${firstIteration.blocks[0]?.executionId || 'unknown'}`, + id: `${idPrefix}subflow-${iterationType}-${iterationContainerId}-${metadataSource.executionId || 'unknown'}`, timestamp: new Date(subflowStartMs).toISOString(), - workflowId: firstIteration.blocks[0]?.workflowId || '', + workflowId: metadataSource.workflowId || '', blockId: `${iterationType}-container-${iterationContainerId}`, blockName: iterationType.charAt(0).toUpperCase() + iterationType.slice(1), blockType: iterationType, - executionId: firstIteration.blocks[0]?.executionId, + executionId: metadataSource.executionId, startedAt: new Date(subflowStartMs).toISOString(), executionOrder: subflowExecutionOrder, endedAt: new Date(subflowEndMs).toISOString(), durationMs: subflowDuration, - success: !allBlocks.some((b) => b.error), + success: !allRelevantBlocks.some((b) => b.error), + iterationContainerId, } - // Build iteration child nodes - const iterationNodes: EntryNode[] = iterationGroups.map((iterGroup) => { - // Create synthetic iteration entry - const iterBlocks = iterGroup.blocks - const iterStartMs = Math.min( - ...iterBlocks.map((b) => new Date(b.startedAt || b.timestamp).getTime()) - ) - const iterEndMs = Math.max( - ...iterBlocks.map((b) => new Date(b.endedAt || b.timestamp).getTime()) - ) - const iterDuration = iterBlocks.reduce((sum, b) => sum + (b.durationMs || 0), 0) - // Parallel branches run concurrently — use wall-clock time. Loop iterations run serially — use sum. - const iterDisplayDuration = - iterationType === 'parallel' ? iterEndMs - iterStartMs : iterDuration + const iterationNodes: EntryNode[] = iterationGroups + .map((iterGroup): EntryNode | null => { + const matchingNestedEntries = nestedForThisSubflow.filter((e) => { + const parent = e.parentIterations?.[0] + return parent?.iterationCurrent === iterGroup.iterationCurrent + }) - // Use the minimum executionOrder from blocks in this iteration - const iterExecutionOrder = Math.min(...iterBlocks.map((b) => b.executionOrder)) - const syntheticIteration: ConsoleEntry = { - id: `iteration-${iterationType}-${iterGroup.iterationContainerId}-${iterGroup.iterationCurrent}-${iterBlocks[0]?.executionId || 'unknown'}`, - timestamp: new Date(iterStartMs).toISOString(), - workflowId: iterBlocks[0]?.workflowId || '', - blockId: `iteration-${iterGroup.iterationContainerId}-${iterGroup.iterationCurrent}`, - blockName: `Iteration ${iterGroup.iterationCurrent}${iterGroup.iterationTotal !== undefined ? ` / ${iterGroup.iterationTotal}` : ''}`, - blockType: iterationType, - executionId: iterBlocks[0]?.executionId, - startedAt: new Date(iterStartMs).toISOString(), - executionOrder: iterExecutionOrder, - endedAt: new Date(iterEndMs).toISOString(), - durationMs: iterDisplayDuration, - success: !iterBlocks.some((b) => b.error), - iterationCurrent: iterGroup.iterationCurrent, - iterationTotal: iterGroup.iterationTotal, - iterationType: iterationType as 'loop' | 'parallel', - iterationContainerId: iterGroup.iterationContainerId, - } + const strippedNestedEntries: ConsoleEntry[] = matchingNestedEntries.map((e) => ({ + ...e, + parentIterations: + e.parentIterations && e.parentIterations.length > 1 + ? e.parentIterations.slice(1) + : undefined, + })) - // Block nodes within this iteration — workflow blocks get their full subtree - const blockNodes: EntryNode[] = iterBlocks.map((block) => { - if (isWorkflowBlockType(block.blockType)) { - const instanceKey = block.childWorkflowInstanceId ?? block.blockId - const allDescendants = collectWorkflowDescendants(instanceKey, workflowChildGroups) - const rawChildren = allDescendants.map((c) => ({ - ...c, - childWorkflowBlockId: - c.childWorkflowBlockId === instanceKey ? undefined : c.childWorkflowBlockId, - })) - return { - entry: block, - children: buildEntryTree(rawChildren), - nodeType: 'workflow' as const, - } + const iterBlocks = iterGroup.blocks + const allIterEntries = [...iterBlocks, ...strippedNestedEntries] + if (allIterEntries.length === 0) return null + + const iterStartMs = Math.min( + ...allIterEntries.map((b) => new Date(b.startedAt || b.timestamp).getTime()) + ) + const iterEndMs = Math.max( + ...allIterEntries.map((b) => new Date(b.endedAt || b.timestamp).getTime()) + ) + const iterDuration = allIterEntries.reduce((sum, b) => sum + (b.durationMs || 0), 0) + const iterDisplayDuration = + iterationType === 'parallel' ? iterEndMs - iterStartMs : iterDuration + + const iterExecutionOrder = Math.min(...allIterEntries.map((b) => b.executionOrder)) + const iterMetadataSource = allIterEntries[0] + const syntheticIteration: ConsoleEntry = { + id: `${idPrefix}iteration-${iterationType}-${iterGroup.iterationContainerId}-${iterGroup.iterationCurrent}-${iterMetadataSource.executionId || 'unknown'}`, + timestamp: new Date(iterStartMs).toISOString(), + workflowId: iterMetadataSource.workflowId || '', + blockId: `iteration-${iterGroup.iterationContainerId}-${iterGroup.iterationCurrent}`, + blockName: `Iteration ${iterGroup.iterationCurrent}${iterGroup.iterationTotal !== undefined ? ` / ${iterGroup.iterationTotal}` : ''}`, + blockType: iterationType, + executionId: iterMetadataSource.executionId, + startedAt: new Date(iterStartMs).toISOString(), + executionOrder: iterExecutionOrder, + endedAt: new Date(iterEndMs).toISOString(), + durationMs: iterDisplayDuration, + success: !allIterEntries.some((b) => b.error), + iterationCurrent: iterGroup.iterationCurrent, + iterationTotal: iterGroup.iterationTotal, + iterationType: iterationType as 'loop' | 'parallel', + iterationContainerId: iterGroup.iterationContainerId, } - return { entry: block, children: [], nodeType: 'block' as const } - }) - return { - entry: syntheticIteration, - children: blockNodes, - nodeType: 'iteration' as const, - iterationInfo: { - current: iterGroup.iterationCurrent, - total: iterGroup.iterationTotal, - }, - } - }) + const childPrefix = `${idPrefix}${iterationContainerId}-${iterGroup.iterationCurrent}-` + const nestedSubflowNodes = + strippedNestedEntries.length > 0 ? buildEntryTree(strippedNestedEntries, childPrefix) : [] + + // Filter out container completion events when matching nested subflow nodes exist, + // to avoid duplicating them as both a flat block row and an expandable subflow. + const hasNestedSubflows = nestedSubflowNodes.length > 0 + const blockNodes: EntryNode[] = iterBlocks + .filter((block) => { + if ( + hasNestedSubflows && + (block.blockType === 'loop' || block.blockType === 'parallel') + ) { + return false + } + return true + }) + .map((block) => { + if (isWorkflowBlockType(block.blockType)) { + const instanceKey = block.childWorkflowInstanceId ?? block.blockId + const allDescendants = collectWorkflowDescendants(instanceKey, workflowChildGroups) + const rawChildren = allDescendants.map((c) => ({ + ...c, + childWorkflowBlockId: + c.childWorkflowBlockId === instanceKey ? undefined : c.childWorkflowBlockId, + })) + return { + entry: block, + children: buildEntryTree(rawChildren), + nodeType: 'workflow' as const, + } + } + return { entry: block, children: [], nodeType: 'block' as const } + }) + + const allChildren = [...blockNodes, ...nestedSubflowNodes] + allChildren.sort((a, b) => a.entry.executionOrder - b.entry.executionOrder) + + return { + entry: syntheticIteration, + children: allChildren, + nodeType: 'iteration' as const, + iterationInfo: { + current: iterGroup.iterationCurrent, + total: iterGroup.iterationTotal, + }, + } + }) + .filter((node): node is EntryNode => node !== null) subflowNodes.push({ entry: syntheticSubflow, @@ -422,7 +513,6 @@ function buildEntryTree(entries: ConsoleEntry[]): EntryNode[] { }) } - // Build workflow nodes for regular blocks that are workflow block types const workflowNodes: EntryNode[] = [] const remainingRegularBlocks: ConsoleEntry[] = [] @@ -442,16 +532,15 @@ function buildEntryTree(entries: ConsoleEntry[]): EntryNode[] { } } - // Build nodes for remaining regular blocks const regularNodes: EntryNode[] = remainingRegularBlocks.map((entry) => ({ entry, children: [], nodeType: 'block' as const, })) - // Combine all nodes and sort by executionOrder ascending (oldest first, top-down) const allNodes = [...subflowNodes, ...workflowNodes, ...regularNodes] allNodes.sort((a, b) => a.entry.executionOrder - b.entry.executionOrder) + return allNodes } diff --git a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/hooks/use-node-utilities.ts b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/hooks/use-node-utilities.ts index 06329a6b71..f8578d95ea 100644 --- a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/hooks/use-node-utilities.ts +++ b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/hooks/use-node-utilities.ts @@ -81,16 +81,43 @@ export function useNodeUtilities(blocks: Record) { * @returns Array of node IDs representing the hierarchy path */ const getNodeHierarchy = useCallback( - (nodeId: string): string[] => { + (nodeId: string, maxDepth = 100): string[] => { const node = getNodes().find((n) => n.id === nodeId) - if (!node) return [nodeId] + if (!node || maxDepth <= 0) return [nodeId] const parentId = blocks?.[nodeId]?.data?.parentId if (!parentId) return [nodeId] - return [...getNodeHierarchy(parentId), nodeId] + return [...getNodeHierarchy(parentId, maxDepth - 1), nodeId] }, [getNodes, blocks] ) + /** + * Returns true if nodeId is in the subtree of ancestorId (i.e. walking from nodeId + * up the parentId chain we reach ancestorId). Used to reject parent assignments that + * would create a cycle (e.g. setting dragged node's parent to a container inside it). + * + * @param ancestorId - Node that might be an ancestor + * @param nodeId - Node to walk from (upward) + * @returns True if ancestorId appears in the parent chain of nodeId + */ + const isDescendantOf = useCallback( + (ancestorId: string, nodeId: string): boolean => { + const visited = new Set() + const maxDepth = 100 + let currentId: string | undefined = nodeId + let depth = 0 + while (currentId && depth < maxDepth) { + if (currentId === ancestorId) return true + if (visited.has(currentId)) return false + visited.add(currentId) + currentId = blocks?.[currentId]?.data?.parentId + depth += 1 + } + return false + }, + [blocks] + ) + /** * Gets the absolute position of a node (accounting for nested parents). * For nodes inside containers, accounts for header and padding offsets. @@ -379,6 +406,7 @@ export function useNodeUtilities(blocks: Record) { return { getNodeDepth, getNodeHierarchy, + isDescendantOf, getNodeAbsolutePosition, calculateRelativePosition, isPointInLoopNode, diff --git a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/hooks/use-workflow-execution.ts b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/hooks/use-workflow-execution.ts index b37f030480..3624f455e2 100644 --- a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/hooks/use-workflow-execution.ts +++ b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/hooks/use-workflow-execution.ts @@ -32,6 +32,7 @@ import type { } from '@/executor/types' import { hasExecutionResult } from '@/executor/utils/errors' import { coerceValue } from '@/executor/utils/start-block' +import { stripCloneSuffixes } from '@/executor/utils/subflow-utils' import { subscriptionKeys } from '@/hooks/queries/subscription' import { useExecutionStream } from '@/hooks/use-execution-stream' import { WorkflowValidationError } from '@/serializer' @@ -347,6 +348,22 @@ export function useWorkflowExecution() { return blockType === 'loop' || blockType === 'parallel' } + /** Extracts iteration and child-workflow fields shared across console entry call sites. */ + const extractIterationFields = ( + data: BlockStartedData | BlockCompletedData | BlockErrorData + ) => ({ + iterationCurrent: data.iterationCurrent, + iterationTotal: data.iterationTotal, + iterationType: data.iterationType, + iterationContainerId: data.iterationContainerId, + parentIterations: data.parentIterations, + childWorkflowBlockId: data.childWorkflowBlockId, + childWorkflowName: data.childWorkflowName, + ...('childWorkflowInstanceId' in data && { + childWorkflowInstanceId: data.childWorkflowInstanceId, + }), + }) + const createBlockLogEntry = ( data: BlockCompletedData | BlockErrorData, options: { success: boolean; output?: unknown; error?: string } @@ -379,13 +396,7 @@ export function useWorkflowExecution() { executionId: executionIdRef.current, blockName: data.blockName || 'Unknown Block', blockType: data.blockType || 'unknown', - iterationCurrent: data.iterationCurrent, - iterationTotal: data.iterationTotal, - iterationType: data.iterationType, - iterationContainerId: data.iterationContainerId, - childWorkflowBlockId: data.childWorkflowBlockId, - childWorkflowName: data.childWorkflowName, - childWorkflowInstanceId: data.childWorkflowInstanceId, + ...extractIterationFields(data), }) } @@ -405,13 +416,7 @@ export function useWorkflowExecution() { executionId: executionIdRef.current, blockName: data.blockName || 'Unknown Block', blockType: data.blockType || 'unknown', - iterationCurrent: data.iterationCurrent, - iterationTotal: data.iterationTotal, - iterationType: data.iterationType, - iterationContainerId: data.iterationContainerId, - childWorkflowBlockId: data.childWorkflowBlockId, - childWorkflowName: data.childWorkflowName, - childWorkflowInstanceId: data.childWorkflowInstanceId, + ...extractIterationFields(data), }) } @@ -427,13 +432,7 @@ export function useWorkflowExecution() { startedAt: data.startedAt, endedAt: data.endedAt, isRunning: false, - iterationCurrent: data.iterationCurrent, - iterationTotal: data.iterationTotal, - iterationType: data.iterationType, - iterationContainerId: data.iterationContainerId, - childWorkflowBlockId: data.childWorkflowBlockId, - childWorkflowName: data.childWorkflowName, - childWorkflowInstanceId: data.childWorkflowInstanceId, + ...extractIterationFields(data), }, executionIdRef.current ) @@ -452,13 +451,7 @@ export function useWorkflowExecution() { startedAt: data.startedAt, endedAt: data.endedAt, isRunning: false, - iterationCurrent: data.iterationCurrent, - iterationTotal: data.iterationTotal, - iterationType: data.iterationType, - iterationContainerId: data.iterationContainerId, - childWorkflowBlockId: data.childWorkflowBlockId, - childWorkflowName: data.childWorkflowName, - childWorkflowInstanceId: data.childWorkflowInstanceId, + ...extractIterationFields(data), }, executionIdRef.current ) @@ -486,12 +479,7 @@ export function useWorkflowExecution() { blockName: data.blockName || 'Unknown Block', blockType: data.blockType || 'unknown', isRunning: true, - iterationCurrent: data.iterationCurrent, - iterationTotal: data.iterationTotal, - iterationType: data.iterationType, - iterationContainerId: data.iterationContainerId, - childWorkflowBlockId: data.childWorkflowBlockId, - childWorkflowName: data.childWorkflowName, + ...extractIterationFields(data), }) } @@ -499,7 +487,6 @@ export function useWorkflowExecution() { if (isStaleExecution()) return updateActiveBlocks(data.blockId, false) if (workflowId) setBlockRunStatus(workflowId, data.blockId, 'success') - executedBlockIds.add(data.blockId) accumulatedBlockStates.set(data.blockId, { output: data.output, @@ -507,7 +494,17 @@ export function useWorkflowExecution() { executionTime: data.durationMs, }) + // For nested containers, the SSE blockId may be a cloned ID (e.g. P1__obranch-0). + // Also record the original workflow-level ID so the canvas can highlight it. if (isContainerBlockType(data.blockType)) { + const originalId = stripCloneSuffixes(data.blockId) + if (originalId !== data.blockId) { + executedBlockIds.add(originalId) + if (workflowId) setBlockRunStatus(workflowId, originalId, 'success') + } + } + + if (isContainerBlockType(data.blockType) && !data.iterationContainerId) { return } @@ -538,6 +535,15 @@ export function useWorkflowExecution() { executionTime: data.durationMs || 0, }) + // For nested containers, also record the original workflow-level ID + if (isContainerBlockType(data.blockType)) { + const originalId = stripCloneSuffixes(data.blockId) + if (originalId !== data.blockId) { + executedBlockIds.add(originalId) + if (workflowId) setBlockRunStatus(workflowId, originalId, 'error') + } + } + accumulatedBlockLogs.push( createBlockLogEntry(data, { success: false, output: {}, error: data.error }) ) diff --git a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/utils/workflow-canvas-helpers.ts b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/utils/workflow-canvas-helpers.ts index 7f24907c47..3306fac0ff 100644 --- a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/utils/workflow-canvas-helpers.ts +++ b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/utils/workflow-canvas-helpers.ts @@ -4,6 +4,39 @@ import { TriggerUtils } from '@/lib/workflows/triggers/triggers' import { clampPositionToContainer } from '@/app/workspace/[workspaceId]/w/[workflowId]/utils/node-position-utils' import type { BlockState } from '@/stores/workflows/workflow/types' +/** + * Collects all descendant block IDs for container blocks (loop/parallel) in the given set. + * Used to treat a nested subflow as one unit when computing boundary edges (e.g. remove-from-subflow). + * + * @param blockIds - Root block IDs (e.g. the blocks being removed from subflow) + * @param blocks - All workflow blocks + * @returns IDs of blocks that are descendants of any container in blockIds (excluding the roots) + */ +export function getDescendantBlockIds( + blockIds: string[], + blocks: Record +): string[] { + const current = new Set(blockIds) + const added: string[] = [] + const toProcess = [...blockIds] + + while (toProcess.length > 0) { + const id = toProcess.pop()! + const block = blocks[id] + if (block?.type !== 'loop' && block?.type !== 'parallel') continue + + for (const [bid, b] of Object.entries(blocks)) { + if (b?.data?.parentId === id && !current.has(bid)) { + current.add(bid) + added.push(bid) + toProcess.push(bid) + } + } + } + + return added +} + /** * Checks if the currently focused element is an editable input. * Returns true if the user is typing in an input, textarea, or contenteditable element. diff --git a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/workflow.tsx b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/workflow.tsx index 02c6175c2f..3b4559fde9 100644 --- a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/workflow.tsx +++ b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/workflow.tsx @@ -57,6 +57,7 @@ import { estimateBlockDimensions, filterProtectedBlocks, getClampedPositionForNode, + getDescendantBlockIds, getWorkflowLockToggleIds, isBlockProtected, isEdgeProtected, @@ -197,7 +198,7 @@ const defaultEdgeOptions = { type: 'custom' } const reactFlowStyles = [ 'bg-[var(--bg)]', '[&_.react-flow__edges]:!z-0', - '[&_.react-flow__node]:!z-[21]', + '[&_.react-flow__node]:z-[21]', '[&_.react-flow__handle]:!z-[30]', '[&_.react-flow__edge-labels]:!z-[60]', '[&_.react-flow__pane]:!bg-[var(--bg)]', @@ -416,6 +417,7 @@ const WorkflowContent = React.memo(() => { const { getNodeDepth, getNodeAbsolutePosition, + isDescendantOf, calculateRelativePosition, isPointInLoopNode, resizeLoopNodes, @@ -432,7 +434,6 @@ const WorkflowContent = React.memo(() => { const canNodeEnterContainer = useCallback( (node: Node): boolean => { if (node.data?.type === 'starter') return false - if (node.type === 'subflowNode') return false const block = blocks[node.id] return !(block && TriggerUtils.isTriggerBlock(block)) }, @@ -681,10 +682,15 @@ const WorkflowContent = React.memo(() => { if (nodesNeedingUpdate.length === 0) return // Filter out nodes that cannot enter containers (when target is a container) - const validNodes = targetParentId + let validNodes = targetParentId ? nodesNeedingUpdate.filter(canNodeEnterContainer) : nodesNeedingUpdate + // Exclude nodes that would create a cycle (moving a container into one of its descendants) + if (targetParentId) { + validNodes = validNodes.filter((n) => !isDescendantOf(n.id, targetParentId)) + } + if (validNodes.length === 0) return // Find boundary edges (edges that cross the container boundary) @@ -744,6 +750,7 @@ const WorkflowContent = React.memo(() => { blocks, edgesForDisplay, canNodeEnterContainer, + isDescendantOf, calculateRelativePosition, getNodeAbsolutePosition, shiftUpdatesToContainerBounds, @@ -1014,12 +1021,22 @@ const WorkflowContent = React.memo(() => { return } - // Check if any pasted block is a subflow - subflows cannot be nested - const hasSubflow = pastedBlocksArray.some((b) => b.type === 'loop' || b.type === 'parallel') - if (hasSubflow) { + // Prevent cycle: pasting a container that is the target container itself or one of its ancestors. + // Use original clipboard IDs since preparePasteData regenerates them via uuidv4(). + const ancestorIds = new Set() + let walkId: string | undefined = targetContainer.loopId + while (walkId && !ancestorIds.has(walkId)) { + ancestorIds.add(walkId) + walkId = blocks[walkId]?.data?.parentId as string | undefined + } + const originalClipboardBlocks = clipboard ? Object.values(clipboard.blocks) : [] + const wouldCreateCycle = originalClipboardBlocks.some( + (b) => (b.type === 'loop' || b.type === 'parallel') && ancestorIds.has(b.id) + ) + if (wouldCreateCycle) { addNotification({ level: 'error', - message: 'Subflows cannot be nested inside other subflows.', + message: 'Cannot paste a subflow inside itself or its own descendant.', workflowId: activeWorkflowId || undefined, }) return @@ -1702,31 +1719,75 @@ const WorkflowContent = React.memo(() => { const containerInfo = isPointInLoopNode(position) clearDragHighlights() - document.body.classList.remove('sim-drag-subflow') if (data.type === 'loop' || data.type === 'parallel') { const id = crypto.randomUUID() const baseName = data.type === 'loop' ? 'Loop' : 'Parallel' const name = getUniqueBlockName(baseName, blocks) - const autoConnectEdge = tryCreateAutoConnectEdge(position, id, { - targetParentId: null, - }) + if (containerInfo) { + const rawPosition = { + x: position.x - containerInfo.loopPosition.x, + y: position.y - containerInfo.loopPosition.y, + } - addBlock( - id, - data.type, - name, - position, - { - width: CONTAINER_DIMENSIONS.DEFAULT_WIDTH, - height: CONTAINER_DIMENSIONS.DEFAULT_HEIGHT, - type: 'subflowNode', - }, - undefined, - undefined, - autoConnectEdge - ) + const relativePosition = clampPositionToContainer( + rawPosition, + containerInfo.dimensions, + { + width: CONTAINER_DIMENSIONS.DEFAULT_WIDTH, + height: CONTAINER_DIMENSIONS.DEFAULT_HEIGHT, + } + ) + + const existingChildBlocks = Object.values(blocks) + .filter((b) => b.data?.parentId === containerInfo.loopId) + .map((b) => ({ id: b.id, type: b.type, position: b.position })) + + const autoConnectEdge = tryCreateAutoConnectEdge(relativePosition, id, { + targetParentId: containerInfo.loopId, + existingChildBlocks, + containerId: containerInfo.loopId, + }) + + addBlock( + id, + data.type, + name, + relativePosition, + { + width: CONTAINER_DIMENSIONS.DEFAULT_WIDTH, + height: CONTAINER_DIMENSIONS.DEFAULT_HEIGHT, + type: 'subflowNode', + parentId: containerInfo.loopId, + extent: 'parent', + }, + containerInfo.loopId, + 'parent', + autoConnectEdge + ) + + resizeLoopNodesWrapper() + } else { + const autoConnectEdge = tryCreateAutoConnectEdge(position, id, { + targetParentId: null, + }) + + addBlock( + id, + data.type, + name, + position, + { + width: CONTAINER_DIMENSIONS.DEFAULT_WIDTH, + height: CONTAINER_DIMENSIONS.DEFAULT_HEIGHT, + type: 'subflowNode', + }, + undefined, + undefined, + autoConnectEdge + ) + } return } @@ -2113,11 +2174,9 @@ const WorkflowContent = React.memo(() => { // Check if hovering over a container node const containerInfo = isPointInLoopNode(position) - // Highlight container if hovering over it and not dragging a subflow - // Subflow drag is marked by body class flag set by toolbar - const isSubflowDrag = document.body.classList.contains('sim-drag-subflow') + // Highlight container if hovering over it - if (containerInfo && !isSubflowDrag) { + if (containerInfo) { const containerNode = getNodes().find((n) => n.id === containerInfo.loopId) if (containerNode?.type === 'subflowNode') { const kind = (containerNode.data as SubflowNodeData)?.kind @@ -2308,6 +2367,13 @@ const WorkflowContent = React.memo(() => { // Handle container nodes differently if (block.type === 'loop' || block.type === 'parallel') { + // Compute nesting depth so children always render above parents + let depth = 0 + let pid = block.data?.parentId as string | undefined + while (pid && depth < 100) { + depth++ + pid = blocks[pid]?.data?.parentId as string | undefined + } nodeArray.push({ id: block.id, type: 'subflowNode', @@ -2316,6 +2382,8 @@ const WorkflowContent = React.memo(() => { extent: block.data?.extent || undefined, dragHandle: '.workflow-drag-handle', draggable: !isBlockProtected(block.id, blocks), + zIndex: depth, + className: block.data?.parentId ? 'nested-subflow-node' : undefined, data: { ...block.data, name: block.name, @@ -2476,15 +2544,35 @@ const WorkflowContent = React.memo(() => { }) if (validBlockIds.length === 0) return - const movingNodeIds = new Set(validBlockIds) + const validBlockIdSet = new Set(validBlockIds) + const descendantIds = getDescendantBlockIds(validBlockIds, blocks) + const movingNodeIds = new Set([...validBlockIds, ...descendantIds]) - // Find boundary edges (edges that cross the subflow boundary) + // Find boundary edges (one end inside the subtree, one end outside) const boundaryEdges = edgesForDisplay.filter((e) => { const sourceInSelection = movingNodeIds.has(e.source) const targetInSelection = movingNodeIds.has(e.target) return sourceInSelection !== targetInSelection }) - const boundaryEdgesByNode = mapEdgesByNode(boundaryEdges, movingNodeIds) + + // Attribute each boundary edge to the validBlockId that is the ancestor of the moved endpoint + const boundaryEdgesByNode = new Map() + for (const edge of boundaryEdges) { + const movedEnd = movingNodeIds.has(edge.source) ? edge.source : edge.target + let id: string | undefined = movedEnd + const seen = new Set() + while (id) { + if (seen.has(id)) break + seen.add(id) + if (validBlockIdSet.has(id)) { + const list = boundaryEdgesByNode.get(id) ?? [] + list.push(edge) + boundaryEdgesByNode.set(id, list) + break + } + id = blocks[id]?.data?.parentId + } + } // Collect absolute positions BEFORE any mutations const absolutePositions = new Map() @@ -2546,42 +2634,54 @@ const WorkflowContent = React.memo(() => { /** * Updates container dimensions in displayNodes during drag or keyboard movement. + * Resizes the moved node's immediate parent and all ancestor containers (for nested loops/parallels). */ const updateContainerDimensionsDuringMove = useCallback( (movedNodeId: string, movedNodePosition: { x: number; y: number }) => { - const parentId = blocks[movedNodeId]?.data?.parentId - if (!parentId) return + const ancestorIds: string[] = [] + const visited = new Set() + let currentId = blocks[movedNodeId]?.data?.parentId + while (currentId && !visited.has(currentId)) { + visited.add(currentId) + ancestorIds.push(currentId) + currentId = blocks[currentId]?.data?.parentId + } + if (ancestorIds.length === 0) return setDisplayNodes((currentNodes) => { - const childNodes = currentNodes.filter((n) => n.parentId === parentId) - if (childNodes.length === 0) return currentNodes + const computedDimensions = new Map() - const childPositions = childNodes.map((node) => { - const nodePosition = node.id === movedNodeId ? movedNodePosition : node.position - const { width, height } = getBlockDimensions(node.id) - return { x: nodePosition.x, y: nodePosition.y, width, height } - }) + for (const containerId of ancestorIds) { + const childNodes = currentNodes.filter((n) => n.parentId === containerId) + if (childNodes.length === 0) continue - const { width: newWidth, height: newHeight } = calculateContainerDimensions(childPositions) + const childPositions = childNodes.map((node) => { + const nodePosition = node.id === movedNodeId ? movedNodePosition : node.position + const dims = computedDimensions.get(node.id) + const width = dims?.width ?? node.data?.width ?? getBlockDimensions(node.id).width + const height = dims?.height ?? node.data?.height ?? getBlockDimensions(node.id).height + return { x: nodePosition.x, y: nodePosition.y, width, height } + }) + + computedDimensions.set(containerId, calculateContainerDimensions(childPositions)) + } return currentNodes.map((node) => { - if (node.id === parentId) { - const currentWidth = node.data?.width || CONTAINER_DIMENSIONS.DEFAULT_WIDTH - const currentHeight = node.data?.height || CONTAINER_DIMENSIONS.DEFAULT_HEIGHT - - // Only update if dimensions changed - if (newWidth !== currentWidth || newHeight !== currentHeight) { - return { - ...node, - data: { - ...node.data, - width: newWidth, - height: newHeight, - }, - } - } + const newDims = computedDimensions.get(node.id) + if (!newDims) return node + const currentWidth = node.data?.width ?? CONTAINER_DIMENSIONS.DEFAULT_WIDTH + const currentHeight = node.data?.height ?? CONTAINER_DIMENSIONS.DEFAULT_HEIGHT + if (newDims.width === currentWidth && newDims.height === currentHeight) { + return node + } + return { + ...node, + data: { + ...node.data, + width: newDims.width, + height: newDims.height, + }, } - return node }) }) }, @@ -2914,16 +3014,6 @@ const WorkflowContent = React.memo(() => { // Get the node's absolute position to properly calculate intersections const nodeAbsolutePos = getNodeAbsolutePosition(node.id) - // Prevent subflows from being dragged into other subflows - if (node.type === 'subflowNode') { - // Clear any highlighting for subflow nodes - if (potentialParentId) { - clearDragHighlights() - setPotentialParentId(null) - } - return // Exit early - subflows cannot be placed inside other subflows - } - // Find intersections with container nodes using absolute coordinates const intersectingNodes = getNodes() .filter((n) => { @@ -2993,15 +3083,25 @@ const WorkflowContent = React.memo(() => { return a.size - b.size // Smaller container takes precedence }) + // Exclude containers that are inside the dragged node (would create a cycle) + const validContainers = sortedContainers.filter( + ({ container }) => !isDescendantOf(node.id, container.id) + ) + // Use the most appropriate container (deepest or smallest at same depth) - const bestContainerMatch = sortedContainers[0] + const bestContainerMatch = validContainers[0] - setPotentialParentId(bestContainerMatch.container.id) + if (bestContainerMatch) { + setPotentialParentId(bestContainerMatch.container.id) - // Add highlight class and change cursor - const kind = (bestContainerMatch.container.data as SubflowNodeData)?.kind - if (kind === 'loop' || kind === 'parallel') { - highlightContainerNode(bestContainerMatch.container.id, kind) + // Add highlight class and change cursor + const kind = (bestContainerMatch.container.data as SubflowNodeData)?.kind + if (kind === 'loop' || kind === 'parallel') { + highlightContainerNode(bestContainerMatch.container.id, kind) + } + } else { + clearDragHighlights() + setPotentialParentId(null) } } else { // Remove highlighting if no longer over a container @@ -3017,6 +3117,7 @@ const WorkflowContent = React.memo(() => { blocks, getNodeAbsolutePosition, getNodeDepth, + isDescendantOf, updateContainerDimensionsDuringMove, highlightContainerNode, ] @@ -3159,6 +3260,17 @@ const WorkflowContent = React.memo(() => { } } + // Prevent placing a container inside one of its own nested containers (would create cycle) + if (potentialParentId && isDescendantOf(node.id, potentialParentId)) { + addNotification({ + level: 'info', + message: 'Cannot place a container inside one of its own nested containers', + workflowId: activeWorkflowId || undefined, + }) + setPotentialParentId(null) + return + } + // Update the node's parent relationship if (potentialParentId) { // Remove existing edges before moving into container @@ -3275,6 +3387,7 @@ const WorkflowContent = React.memo(() => { getNodes, dragStartParentId, potentialParentId, + isDescendantOf, updateNodeParent, updateBlockPosition, collaborativeBatchAddEdges, diff --git a/apps/sim/app/workspace/[workspaceId]/w/components/preview/components/preview-workflow/preview-workflow.tsx b/apps/sim/app/workspace/[workspaceId]/w/components/preview/components/preview-workflow/preview-workflow.tsx index cdad585440..93dc49ffab 100644 --- a/apps/sim/app/workspace/[workspaceId]/w/components/preview/components/preview-workflow/preview-workflow.tsx +++ b/apps/sim/app/workspace/[workspaceId]/w/components/preview/components/preview-workflow/preview-workflow.tsx @@ -289,21 +289,38 @@ export function PreviewWorkflow({ return map }, [executedBlocks]) - /** Derives subflow status from children. Error takes precedence. */ + /** Derives subflow status from children. Recursively checks nested subflows. Error takes precedence. */ const getSubflowExecutionStatus = useMemo(() => { - return (subflowId: string): ExecutionStatus | undefined => { + const derive = ( + subflowId: string, + visited: Set = new Set() + ): ExecutionStatus | undefined => { + if (visited.has(subflowId)) return undefined + visited.add(subflowId) + const childIds = subflowChildrenMap.get(subflowId) if (!childIds?.length) return undefined - const executedChildren = childIds - .map((id) => blockExecutionMap.get(id)) - .filter((status): status is { status: string } => Boolean(status)) + const childStatuses: string[] = [] + for (const childId of childIds) { + const direct = blockExecutionMap.get(childId) + if (direct) { + childStatuses.push(direct.status) + } else { + const childBlock = workflowState.blocks?.[childId] + if (childBlock?.type === 'loop' || childBlock?.type === 'parallel') { + const nested = derive(childId, visited) + if (nested) childStatuses.push(nested) + } + } + } - if (executedChildren.length === 0) return undefined - if (executedChildren.some((s) => s.status === 'error')) return 'error' + if (childStatuses.length === 0) return undefined + if (childStatuses.some((s) => s === 'error')) return 'error' return 'success' } - }, [subflowChildrenMap, blockExecutionMap]) + return derive + }, [subflowChildrenMap, blockExecutionMap, workflowState.blocks]) /** Gets block status. Subflows derive status from children. */ const getBlockExecutionStatus = useMemo(() => { diff --git a/apps/sim/components/icons.tsx b/apps/sim/components/icons.tsx index 22c5b5718e..5525e048cf 100644 --- a/apps/sim/components/icons.tsx +++ b/apps/sim/components/icons.tsx @@ -1711,167 +1711,31 @@ export function StagehandIcon(props: SVGProps) { return ( - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ) diff --git a/apps/sim/executor/constants.ts b/apps/sim/executor/constants.ts index 90c632fa5f..65dafebd8d 100644 --- a/apps/sim/executor/constants.ts +++ b/apps/sim/executor/constants.ts @@ -158,6 +158,8 @@ export const DEFAULTS = { MAX_LOOP_ITERATIONS: 1000, MAX_FOREACH_ITEMS: 1000, MAX_PARALLEL_BRANCHES: 20, + MAX_NESTING_DEPTH: 10, + /** Maximum child workflow depth for propagating SSE callbacks (block:started, block:completed). */ MAX_SSE_CHILD_DEPTH: 3, EXECUTION_TIME: 0, TOKENS: { diff --git a/apps/sim/executor/dag/builder.test.ts b/apps/sim/executor/dag/builder.test.ts index 39df0681a0..cf3833767b 100644 --- a/apps/sim/executor/dag/builder.test.ts +++ b/apps/sim/executor/dag/builder.test.ts @@ -2,6 +2,11 @@ import { loggerMock } from '@sim/testing' import { describe, expect, it, vi } from 'vitest' import { BlockType } from '@/executor/constants' import { DAGBuilder } from '@/executor/dag/builder' +import { + buildBranchNodeId, + buildParallelSentinelEndId, + buildParallelSentinelStartId, +} from '@/executor/utils/subflow-utils' import type { SerializedBlock, SerializedWorkflow } from '@/serializer/types' vi.mock('@sim/logger', () => loggerMock) @@ -89,6 +94,96 @@ describe('DAGBuilder disabled subflow validation', () => { }) }) +describe('DAGBuilder nested parallel support', () => { + it('builds DAG for parallel-in-parallel with correct sentinel wiring', () => { + const outerParallelId = 'outer-parallel' + const innerParallelId = 'inner-parallel' + const functionId = 'func-1' + + const workflow: SerializedWorkflow = { + version: '1', + blocks: [ + createBlock('start', BlockType.STARTER), + createBlock(outerParallelId, BlockType.PARALLEL), + createBlock(innerParallelId, BlockType.PARALLEL), + createBlock(functionId, BlockType.FUNCTION), + ], + connections: [ + { source: 'start', target: outerParallelId }, + { + source: outerParallelId, + target: innerParallelId, + sourceHandle: 'parallel-start-source', + }, + { + source: innerParallelId, + target: functionId, + sourceHandle: 'parallel-start-source', + }, + ], + loops: {}, + parallels: { + [innerParallelId]: { + id: innerParallelId, + nodes: [functionId], + count: 5, + parallelType: 'count', + }, + [outerParallelId]: { + id: outerParallelId, + nodes: [innerParallelId], + count: 5, + parallelType: 'count', + }, + }, + } + + const builder = new DAGBuilder() + const dag = builder.build(workflow) + + // Outer parallel sentinel pair exists + const outerStartId = buildParallelSentinelStartId(outerParallelId) + const outerEndId = buildParallelSentinelEndId(outerParallelId) + expect(dag.nodes.has(outerStartId)).toBe(true) + expect(dag.nodes.has(outerEndId)).toBe(true) + + // Inner parallel sentinel pair exists + const innerStartId = buildParallelSentinelStartId(innerParallelId) + const innerEndId = buildParallelSentinelEndId(innerParallelId) + expect(dag.nodes.has(innerStartId)).toBe(true) + expect(dag.nodes.has(innerEndId)).toBe(true) + + // Function 1 branch template node exists + const funcTemplateId = buildBranchNodeId(functionId, 0) + expect(dag.nodes.has(funcTemplateId)).toBe(true) + + // Start → outer-sentinel-start + const startNode = dag.nodes.get('start')! + const startTargets = Array.from(startNode.outgoingEdges.values()).map((e) => e.target) + expect(startTargets).toContain(outerStartId) + + // Outer-sentinel-start → inner-sentinel-start + const outerStart = dag.nodes.get(outerStartId)! + const outerStartTargets = Array.from(outerStart.outgoingEdges.values()).map((e) => e.target) + expect(outerStartTargets).toContain(innerStartId) + + // Inner-sentinel-start → function branch template + const innerStart = dag.nodes.get(innerStartId)! + const innerStartTargets = Array.from(innerStart.outgoingEdges.values()).map((e) => e.target) + expect(innerStartTargets).toContain(funcTemplateId) + + // Function branch template → inner-sentinel-end + const funcTemplate = dag.nodes.get(funcTemplateId)! + const funcTargets = Array.from(funcTemplate.outgoingEdges.values()).map((e) => e.target) + expect(funcTargets).toContain(innerEndId) + + // Inner-sentinel-end → outer-sentinel-end + const innerEnd = dag.nodes.get(innerEndId)! + const innerEndTargets = Array.from(innerEnd.outgoingEdges.values()).map((e) => e.target) + expect(innerEndTargets).toContain(outerEndId) + }) +}) + describe('DAGBuilder human-in-the-loop transformation', () => { it('creates trigger nodes and rewires edges for pause blocks', () => { const workflow: SerializedWorkflow = { diff --git a/apps/sim/executor/dag/builder.ts b/apps/sim/executor/dag/builder.ts index f2a43917d3..9064707579 100644 --- a/apps/sim/executor/dag/builder.ts +++ b/apps/sim/executor/dag/builder.ts @@ -8,7 +8,7 @@ import type { DAGEdge, NodeMetadata } from '@/executor/dag/types' import { buildParallelSentinelStartId, buildSentinelStartId, - extractBaseBlockId, + normalizeNodeId, } from '@/executor/utils/subflow-utils' import type { SerializedBlock, @@ -156,7 +156,7 @@ export class DAGBuilder { } const hasConnections = Array.from(sentinelStartNode.outgoingEdges.values()).some((edge) => - nodes.includes(extractBaseBlockId(edge.target)) + nodes.includes(normalizeNodeId(edge.target)) ) if (!hasConnections) { diff --git a/apps/sim/executor/dag/construction/edges.test.ts b/apps/sim/executor/dag/construction/edges.test.ts index a04e784258..e73446a62e 100644 --- a/apps/sim/executor/dag/construction/edges.test.ts +++ b/apps/sim/executor/dag/construction/edges.test.ts @@ -1102,4 +1102,595 @@ describe('EdgeConstructor', () => { }) }) }) + + describe('Nested loop wiring', () => { + it('should wire inner loop sentinels into outer loop sentinel chain', () => { + const outerLoopId = 'outer-loop' + const innerLoopId = 'inner-loop' + const functionId = 'func-1' + const innerFunctionId = 'func-2' + + const outerSentinelStart = `loop-${outerLoopId}-sentinel-start` + const outerSentinelEnd = `loop-${outerLoopId}-sentinel-end` + const innerSentinelStart = `loop-${innerLoopId}-sentinel-start` + const innerSentinelEnd = `loop-${innerLoopId}-sentinel-end` + + const outerLoop: SerializedLoop = { + id: outerLoopId, + nodes: [functionId, innerLoopId], + iterations: 5, + loopType: 'for', + } + const innerLoop: SerializedLoop = { + id: innerLoopId, + nodes: [innerFunctionId], + iterations: 3, + loopType: 'for', + } + + const dag = createMockDAG([ + functionId, + innerFunctionId, + outerSentinelStart, + outerSentinelEnd, + innerSentinelStart, + innerSentinelEnd, + ]) + dag.loopConfigs.set(outerLoopId, outerLoop) + dag.loopConfigs.set(innerLoopId, innerLoop) + + const workflow = createMockWorkflow( + [ + createMockBlock(functionId), + createMockBlock(innerFunctionId), + createMockBlock(innerLoopId, 'loop'), + ], + [{ source: functionId, target: innerLoopId }], + { [outerLoopId]: outerLoop, [innerLoopId]: innerLoop } + ) + + edgeConstructor.execute( + workflow, + dag, + new Set(), + new Set([functionId, innerLoopId, innerFunctionId]), + new Set([ + functionId, + innerFunctionId, + innerLoopId, + outerSentinelStart, + outerSentinelEnd, + innerSentinelStart, + innerSentinelEnd, + ]), + new Map() + ) + + const outerStartNode = dag.nodes.get(outerSentinelStart)! + const outerStartTargets = Array.from(outerStartNode.outgoingEdges.values()).map( + (e) => e.target + ) + expect(outerStartTargets).toContain(functionId) + + const funcNode = dag.nodes.get(functionId)! + const funcTargets = Array.from(funcNode.outgoingEdges.values()).map((e) => e.target) + expect(funcTargets).toContain(innerSentinelStart) + + const innerEndNode = dag.nodes.get(innerSentinelEnd)! + const innerEndEdges = Array.from(innerEndNode.outgoingEdges.values()) + const exitEdge = innerEndEdges.find((e) => e.target === outerSentinelEnd) + expect(exitEdge).toBeDefined() + expect(exitEdge!.sourceHandle).toBe('loop_exit') + + const backEdge = innerEndEdges.find((e) => e.target === innerSentinelStart) + expect(backEdge).toBeDefined() + expect(backEdge!.sourceHandle).toBe('loop_continue') + + const outerEndNode = dag.nodes.get(outerSentinelEnd)! + const outerBackEdge = Array.from(outerEndNode.outgoingEdges.values()).find( + (e) => e.target === outerSentinelStart + ) + expect(outerBackEdge).toBeDefined() + expect(outerBackEdge!.sourceHandle).toBe('loop_continue') + }) + + it('should correctly identify boundary nodes when inner loop is the only node', () => { + const outerLoopId = 'outer-loop' + const innerLoopId = 'inner-loop' + const innerFunctionId = 'func-inner' + + const outerSentinelStart = `loop-${outerLoopId}-sentinel-start` + const outerSentinelEnd = `loop-${outerLoopId}-sentinel-end` + const innerSentinelStart = `loop-${innerLoopId}-sentinel-start` + const innerSentinelEnd = `loop-${innerLoopId}-sentinel-end` + + const outerLoop: SerializedLoop = { + id: outerLoopId, + nodes: [innerLoopId], + iterations: 2, + loopType: 'for', + } + const innerLoop: SerializedLoop = { + id: innerLoopId, + nodes: [innerFunctionId], + iterations: 3, + loopType: 'for', + } + + const dag = createMockDAG([ + innerFunctionId, + outerSentinelStart, + outerSentinelEnd, + innerSentinelStart, + innerSentinelEnd, + ]) + dag.loopConfigs.set(outerLoopId, outerLoop) + dag.loopConfigs.set(innerLoopId, innerLoop) + + const workflow = createMockWorkflow( + [createMockBlock(innerFunctionId), createMockBlock(innerLoopId, 'loop')], + [], + { [outerLoopId]: outerLoop, [innerLoopId]: innerLoop } + ) + + edgeConstructor.execute( + workflow, + dag, + new Set(), + new Set([innerLoopId, innerFunctionId]), + new Set([ + innerFunctionId, + innerLoopId, + outerSentinelStart, + outerSentinelEnd, + innerSentinelStart, + innerSentinelEnd, + ]), + new Map() + ) + + const outerStartNode = dag.nodes.get(outerSentinelStart)! + const outerStartTargets = Array.from(outerStartNode.outgoingEdges.values()).map( + (e) => e.target + ) + expect(outerStartTargets).toContain(innerSentinelStart) + + const innerEndNode = dag.nodes.get(innerSentinelEnd)! + const exitEdge = Array.from(innerEndNode.outgoingEdges.values()).find( + (e) => e.target === outerSentinelEnd + ) + expect(exitEdge).toBeDefined() + expect(exitEdge!.sourceHandle).toBe('loop_exit') + }) + + it('should not drop intra-loop edges when target is a nested loop block', () => { + const outerLoopId = 'outer-loop' + const innerLoopId = 'inner-loop' + const functionId = 'func-1' + const innerFunctionId = 'func-2' + + const outerSentinelStart = `loop-${outerLoopId}-sentinel-start` + const outerSentinelEnd = `loop-${outerLoopId}-sentinel-end` + const innerSentinelStart = `loop-${innerLoopId}-sentinel-start` + const innerSentinelEnd = `loop-${innerLoopId}-sentinel-end` + + const outerLoop: SerializedLoop = { + id: outerLoopId, + nodes: [functionId, innerLoopId], + iterations: 5, + loopType: 'for', + } + const innerLoop: SerializedLoop = { + id: innerLoopId, + nodes: [innerFunctionId], + iterations: 3, + loopType: 'for', + } + + const dag = createMockDAG([ + functionId, + innerFunctionId, + outerSentinelStart, + outerSentinelEnd, + innerSentinelStart, + innerSentinelEnd, + ]) + dag.loopConfigs.set(outerLoopId, outerLoop) + dag.loopConfigs.set(innerLoopId, innerLoop) + + const workflow = createMockWorkflow( + [ + createMockBlock(functionId), + createMockBlock(innerFunctionId), + createMockBlock(innerLoopId, 'loop'), + ], + [{ source: functionId, target: innerLoopId }], + { [outerLoopId]: outerLoop, [innerLoopId]: innerLoop } + ) + + edgeConstructor.execute( + workflow, + dag, + new Set(), + new Set([functionId, innerLoopId, innerFunctionId]), + new Set([ + functionId, + innerFunctionId, + innerLoopId, + outerSentinelStart, + outerSentinelEnd, + innerSentinelStart, + innerSentinelEnd, + ]), + new Map() + ) + + const funcNode = dag.nodes.get(functionId)! + const edgeToInnerStart = Array.from(funcNode.outgoingEdges.values()).find( + (e) => e.target === innerSentinelStart + ) + expect(edgeToInnerStart).toBeDefined() + + const innerStartNode = dag.nodes.get(innerSentinelStart)! + expect(innerStartNode.incomingEdges.has(functionId)).toBe(true) + }) + }) + + describe('Nested parallel wiring', () => { + it('should wire inner parallel sentinels into outer parallel sentinel chain', () => { + const outerParallelId = 'outer-parallel' + const innerParallelId = 'inner-parallel' + const functionId = 'func-1' + + const outerSentinelStart = `parallel-${outerParallelId}-sentinel-start` + const outerSentinelEnd = `parallel-${outerParallelId}-sentinel-end` + const innerSentinelStart = `parallel-${innerParallelId}-sentinel-start` + const innerSentinelEnd = `parallel-${innerParallelId}-sentinel-end` + const funcTemplate = `${functionId}₍0₎` + + const dag = createMockDAG([ + outerSentinelStart, + outerSentinelEnd, + innerSentinelStart, + innerSentinelEnd, + funcTemplate, + ]) + + // Set up sentinel metadata + dag.nodes.get(outerSentinelStart)!.metadata = { + isSentinel: true, + isParallelSentinel: true, + sentinelType: 'start', + parallelId: outerParallelId, + } + dag.nodes.get(outerSentinelEnd)!.metadata = { + isSentinel: true, + isParallelSentinel: true, + sentinelType: 'end', + parallelId: outerParallelId, + } + dag.nodes.get(innerSentinelStart)!.metadata = { + isSentinel: true, + isParallelSentinel: true, + sentinelType: 'start', + parallelId: innerParallelId, + } + dag.nodes.get(innerSentinelEnd)!.metadata = { + isSentinel: true, + isParallelSentinel: true, + sentinelType: 'end', + parallelId: innerParallelId, + } + dag.nodes.get(funcTemplate)!.metadata = { + isParallelBranch: true, + parallelId: innerParallelId, + branchIndex: 0, + branchTotal: 1, + originalBlockId: functionId, + } + + dag.parallelConfigs.set(outerParallelId, { + id: outerParallelId, + nodes: [innerParallelId], + count: 3, + parallelType: 'count', + }) + dag.parallelConfigs.set(innerParallelId, { + id: innerParallelId, + nodes: [functionId], + count: 2, + parallelType: 'count', + }) + + const workflow = createMockWorkflow( + [createMockBlock(functionId)], + [ + // Outer parallel start → inner parallel (intra-parallel, skipped by wireRegularEdges) + { + source: outerParallelId, + target: innerParallelId, + sourceHandle: 'parallel-start-source', + }, + // Inner parallel start → function (intra-parallel, skipped by wireRegularEdges) + { + source: innerParallelId, + target: functionId, + sourceHandle: 'parallel-start-source', + }, + ] + ) + + const edgeConstructor = new EdgeConstructor() + edgeConstructor.execute( + workflow, + dag, + new Set([innerParallelId, functionId]), + new Set(), + new Set([outerParallelId, innerParallelId, functionId]), + new Map() + ) + + // Outer sentinel-start → inner sentinel-start + const outerStartNode = dag.nodes.get(outerSentinelStart)! + const edgeToInnerStart = Array.from(outerStartNode.outgoingEdges.values()).find( + (e) => e.target === innerSentinelStart + ) + expect(edgeToInnerStart).toBeDefined() + + // Inner sentinel-end → outer sentinel-end + const innerEndNode = dag.nodes.get(innerSentinelEnd)! + const edgeToOuterEnd = Array.from(innerEndNode.outgoingEdges.values()).find( + (e) => e.target === outerSentinelEnd + ) + expect(edgeToOuterEnd).toBeDefined() + + // Inner sentinel-start → func template + const innerStartNode = dag.nodes.get(innerSentinelStart)! + const edgeToFunc = Array.from(innerStartNode.outgoingEdges.values()).find( + (e) => e.target === funcTemplate + ) + expect(edgeToFunc).toBeDefined() + + // Func template → inner sentinel-end + const funcNode = dag.nodes.get(funcTemplate)! + const edgeToInnerEnd = Array.from(funcNode.outgoingEdges.values()).find( + (e) => e.target === innerSentinelEnd + ) + expect(edgeToInnerEnd).toBeDefined() + }) + + it('should wire parallel-in-loop sentinels correctly', () => { + const loopId = 'outer-loop' + const innerParallelId = 'inner-parallel' + const functionId = 'func-1' + + const loopSentinelStart = `loop-${loopId}-sentinel-start` + const loopSentinelEnd = `loop-${loopId}-sentinel-end` + const parallelSentinelStart = `parallel-${innerParallelId}-sentinel-start` + const parallelSentinelEnd = `parallel-${innerParallelId}-sentinel-end` + const funcTemplate = `${functionId}₍0₎` + + const dag = createMockDAG([ + loopSentinelStart, + loopSentinelEnd, + parallelSentinelStart, + parallelSentinelEnd, + funcTemplate, + ]) + + dag.nodes.get(loopSentinelStart)!.metadata = { + isSentinel: true, + sentinelType: 'start', + loopId, + } + dag.nodes.get(loopSentinelEnd)!.metadata = { + isSentinel: true, + sentinelType: 'end', + loopId, + } + dag.nodes.get(parallelSentinelStart)!.metadata = { + isSentinel: true, + isParallelSentinel: true, + sentinelType: 'start', + parallelId: innerParallelId, + } + dag.nodes.get(parallelSentinelEnd)!.metadata = { + isSentinel: true, + isParallelSentinel: true, + sentinelType: 'end', + parallelId: innerParallelId, + } + dag.nodes.get(funcTemplate)!.metadata = { + isParallelBranch: true, + parallelId: innerParallelId, + branchIndex: 0, + branchTotal: 1, + originalBlockId: functionId, + } + + const outerLoop: SerializedLoop = { + id: loopId, + nodes: [innerParallelId], + iterations: 5, + loopType: 'for', + } + + dag.loopConfigs.set(loopId, outerLoop) + dag.parallelConfigs.set(innerParallelId, { + id: innerParallelId, + nodes: [functionId], + count: 2, + parallelType: 'count', + }) + + const workflow = createMockWorkflow( + [createMockBlock(functionId)], + [ + { + source: loopId, + target: innerParallelId, + sourceHandle: 'loop-start-source', + }, + { + source: innerParallelId, + target: functionId, + sourceHandle: 'parallel-start-source', + }, + ] + ) + + const edgeConstructor = new EdgeConstructor() + edgeConstructor.execute( + workflow, + dag, + new Set([functionId]), + new Set([innerParallelId]), + new Set([loopId, innerParallelId, functionId]), + new Map() + ) + + // Loop sentinel-start → parallel sentinel-start + const loopStartNode = dag.nodes.get(loopSentinelStart)! + const edgeToParallelStart = Array.from(loopStartNode.outgoingEdges.values()).find( + (e) => e.target === parallelSentinelStart + ) + expect(edgeToParallelStart).toBeDefined() + + // Parallel sentinel-end → loop sentinel-end + const parallelEndNode = dag.nodes.get(parallelSentinelEnd)! + const edgeToLoopEnd = Array.from(parallelEndNode.outgoingEdges.values()).find( + (e) => e.target === loopSentinelEnd + ) + expect(edgeToLoopEnd).toBeDefined() + + // Inner parallel wiring: sentinel-start → func, func → sentinel-end + const parallelStartNode = dag.nodes.get(parallelSentinelStart)! + expect( + Array.from(parallelStartNode.outgoingEdges.values()).some((e) => e.target === funcTemplate) + ).toBe(true) + + const funcNode = dag.nodes.get(funcTemplate)! + expect( + Array.from(funcNode.outgoingEdges.values()).some((e) => e.target === parallelSentinelEnd) + ).toBe(true) + }) + + it('should wire loop-in-parallel with correct exit handles', () => { + const outerParallelId = 'outer-parallel' + const innerLoopId = 'inner-loop' + const functionId = 'func-1' + + const outerSentinelStart = `parallel-${outerParallelId}-sentinel-start` + const outerSentinelEnd = `parallel-${outerParallelId}-sentinel-end` + const innerSentinelStart = `loop-${innerLoopId}-sentinel-start` + const innerSentinelEnd = `loop-${innerLoopId}-sentinel-end` + + const dag = createMockDAG([ + outerSentinelStart, + outerSentinelEnd, + innerSentinelStart, + innerSentinelEnd, + functionId, + ]) + + dag.nodes.get(outerSentinelStart)!.metadata = { + isSentinel: true, + isParallelSentinel: true, + sentinelType: 'start', + parallelId: outerParallelId, + } + dag.nodes.get(outerSentinelEnd)!.metadata = { + isSentinel: true, + isParallelSentinel: true, + sentinelType: 'end', + parallelId: outerParallelId, + } + dag.nodes.get(innerSentinelStart)!.metadata = { + isSentinel: true, + sentinelType: 'start', + loopId: innerLoopId, + } + dag.nodes.get(innerSentinelEnd)!.metadata = { + isSentinel: true, + sentinelType: 'end', + loopId: innerLoopId, + } + + const innerLoop: SerializedLoop = { + id: innerLoopId, + nodes: [functionId], + iterations: 3, + loopType: 'for', + } + + dag.loopConfigs.set(innerLoopId, innerLoop) + dag.parallelConfigs.set(outerParallelId, { + id: outerParallelId, + nodes: [innerLoopId], + count: 2, + parallelType: 'count', + }) + + const workflow = createMockWorkflow( + [createMockBlock(functionId), createMockBlock(innerLoopId, 'loop')], + [ + { + source: outerParallelId, + target: innerLoopId, + sourceHandle: 'parallel-start-source', + }, + { + source: innerLoopId, + target: functionId, + sourceHandle: 'loop-start-source', + }, + ], + { [innerLoopId]: innerLoop } + ) + + const edgeConstructor = new EdgeConstructor() + edgeConstructor.execute( + workflow, + dag, + new Set([innerLoopId]), + new Set([functionId]), + new Set([outerParallelId, innerLoopId, functionId]), + new Map() + ) + + // Outer sentinel-start → inner loop sentinel-start + const outerStartNode = dag.nodes.get(outerSentinelStart)! + const edgeToInnerStart = Array.from(outerStartNode.outgoingEdges.values()).find( + (e) => e.target === innerSentinelStart + ) + expect(edgeToInnerStart).toBeDefined() + + // Inner loop sentinel-end → outer parallel sentinel-end with loop_exit handle + const innerEndNode = dag.nodes.get(innerSentinelEnd)! + const edgeToOuterEnd = Array.from(innerEndNode.outgoingEdges.values()).find( + (e) => e.target === outerSentinelEnd + ) + expect(edgeToOuterEnd).toBeDefined() + expect(edgeToOuterEnd!.sourceHandle).toBe('loop_exit') + + // Inner loop back-edge: sentinel-end → sentinel-start with loop_continue handle + const backEdge = Array.from(innerEndNode.outgoingEdges.values()).find( + (e) => e.target === innerSentinelStart + ) + expect(backEdge).toBeDefined() + expect(backEdge!.sourceHandle).toBe('loop_continue') + + // Inner loop wiring: sentinel-start → function + const innerStartNode = dag.nodes.get(innerSentinelStart)! + expect( + Array.from(innerStartNode.outgoingEdges.values()).some((e) => e.target === functionId) + ).toBe(true) + + // Function → inner loop sentinel-end + const funcNode = dag.nodes.get(functionId)! + expect( + Array.from(funcNode.outgoingEdges.values()).some((e) => e.target === innerSentinelEnd) + ).toBe(true) + }) + }) }) diff --git a/apps/sim/executor/dag/construction/edges.ts b/apps/sim/executor/dag/construction/edges.ts index ef6c238de6..f6fd1c8361 100644 --- a/apps/sim/executor/dag/construction/edges.ts +++ b/apps/sim/executor/dag/construction/edges.ts @@ -5,14 +5,14 @@ import { isRouterBlockType, isRouterV2BlockType, } from '@/executor/constants' -import type { DAG } from '@/executor/dag/builder' +import type { DAG, DAGNode } from '@/executor/dag/builder' import { buildBranchNodeId, buildParallelSentinelEndId, buildParallelSentinelStartId, buildSentinelEndId, buildSentinelStartId, - extractBaseBlockId, + normalizeNodeId, } from '@/executor/utils/subflow-utils' import type { SerializedWorkflow } from '@/serializer/types' @@ -62,7 +62,7 @@ export class EdgeConstructor { pauseTriggerMapping ) - this.wireLoopSentinels(dag, reachableBlocks) + this.wireLoopSentinels(dag) this.wireParallelSentinels(dag) } @@ -242,6 +242,11 @@ export class EdgeConstructor { } if (sourceIsParallelBlock) { + // Skip intra-parallel edges (start → child); handled by wireParallelSentinels + const sourceParallelNodes = dag.parallelConfigs.get(originalSource)?.nodes + if (sourceParallelNodes?.includes(originalTarget)) { + continue + } const sentinelEndId = buildParallelSentinelEndId(originalSource) if (!dag.nodes.has(sentinelEndId)) { continue @@ -258,11 +263,12 @@ export class EdgeConstructor { target = sentinelStartId } - if (this.edgeCrossesLoopBoundary(source, target, blocksInLoops, dag)) { + if (this.edgeCrossesLoopBoundary(originalSource, originalTarget, blocksInLoops, dag)) { continue } - if (loopSentinelStartId && !blocksInLoops.has(originalTarget)) { + const sourceLoopNodes = dag.loopConfigs.get(originalSource)?.nodes + if (loopSentinelStartId && !sourceLoopNodes?.includes(originalTarget)) { this.addEdge(dag, loopSentinelStartId, target, EDGE.LOOP_EXIT, targetHandle) } @@ -288,7 +294,7 @@ export class EdgeConstructor { } } - private wireLoopSentinels(dag: DAG, reachableBlocks: Set): void { + private wireLoopSentinels(dag: DAG): void { for (const [loopId, loopConfig] of dag.loopConfigs) { const nodes = loopConfig.nodes @@ -301,14 +307,27 @@ export class EdgeConstructor { continue } - const { startNodes, terminalNodes } = this.findLoopBoundaryNodes(nodes, dag, reachableBlocks) + const { startNodes, terminalNodes } = this.findLoopBoundaryNodes(nodes, dag) for (const startNodeId of startNodes) { - this.addEdge(dag, sentinelStartId, startNodeId) + const resolvedId = this.resolveLoopBlockToSentinelStart(startNodeId, dag) + this.addEdge(dag, sentinelStartId, resolvedId) } for (const terminalNodeId of terminalNodes) { - this.addEdge(dag, terminalNodeId, sentinelEndId) + const resolvedId = this.resolveLoopBlockToSentinelEnd(terminalNodeId, dag) + if (resolvedId !== terminalNodeId) { + // Use the sourceHandle that matches the nested subflow's exit route. + // Parallel sentinel-end outputs selectedRoute "parallel_exit", + // loop sentinel-end outputs "loop_exit". The edge manager only activates + // edges whose sourceHandle matches the source node's selectedRoute. + const handle = dag.parallelConfigs.has(terminalNodeId) + ? EDGE.PARALLEL_EXIT + : EDGE.LOOP_EXIT + this.addEdge(dag, resolvedId, sentinelEndId, handle) + } else { + this.addEdge(dag, resolvedId, sentinelEndId) + } } this.addEdge(dag, sentinelEndId, sentinelStartId, EDGE.LOOP_CONTINUE, undefined, true) @@ -331,21 +350,60 @@ export class EdgeConstructor { const { entryNodes, terminalNodes } = this.findParallelBoundaryNodes(nodes, dag) for (const entryNodeId of entryNodes) { - const templateNodeId = buildBranchNodeId(entryNodeId, 0) - if (dag.nodes.has(templateNodeId)) { - this.addEdge(dag, sentinelStartId, templateNodeId) + const targetId = this.resolveSubflowToSentinelStart(entryNodeId, dag) + if (dag.nodes.has(targetId)) { + this.addEdge(dag, sentinelStartId, targetId) } } for (const terminalNodeId of terminalNodes) { - const templateNodeId = buildBranchNodeId(terminalNodeId, 0) - if (dag.nodes.has(templateNodeId)) { - this.addEdge(dag, templateNodeId, sentinelEndId) + const sourceId = this.resolveSubflowToSentinelEnd(terminalNodeId, dag) + if (dag.nodes.has(sourceId)) { + // Use the sourceHandle that matches the nested subflow's exit route. + // A nested loop sentinel-end outputs "loop_exit", not "parallel_exit". + const handle = dag.loopConfigs.has(terminalNodeId) ? EDGE.LOOP_EXIT : EDGE.PARALLEL_EXIT + this.addEdge(dag, sourceId, sentinelEndId, handle) } } } } + /** + * Resolves a node ID to the appropriate entry point for sentinel wiring. + * Nested parallels → their sentinel-start, nested loops → their sentinel-start, + * regular blocks → their branch template node. + */ + private resolveSubflowToSentinelStart(nodeId: string, dag: DAG): string { + if (dag.parallelConfigs.has(nodeId)) { + return buildParallelSentinelStartId(nodeId) + } + if (dag.loopConfigs.has(nodeId)) { + return buildSentinelStartId(nodeId) + } + return buildBranchNodeId(nodeId, 0) + } + + /** + * Resolves a node ID to the appropriate exit point for sentinel wiring. + * Nested parallels → their sentinel-end, nested loops → their sentinel-end, + * regular blocks → their branch template node. + */ + private resolveSubflowToSentinelEnd(nodeId: string, dag: DAG): string { + if (dag.parallelConfigs.has(nodeId)) { + return buildParallelSentinelEndId(nodeId) + } + if (dag.loopConfigs.has(nodeId)) { + return buildSentinelEndId(nodeId) + } + return buildBranchNodeId(nodeId, 0) + } + + /** + * Checks whether an edge crosses a loop boundary (source and target are in + * different loops, or one is inside a loop and the other is not). Uses the + * original block IDs (pre-sentinel-remapping) because `blocksInLoops` and + * `loopConfigs.nodes` reference original block IDs from the serialized workflow. + */ private edgeCrossesLoopBoundary( source: string, target: string, @@ -363,22 +421,37 @@ export class EdgeConstructor { return false } - let sourceLoopId: string | undefined - let targetLoopId: string | undefined - - for (const [loopId, loopConfig] of dag.loopConfigs) { - if (loopConfig.nodes.includes(source)) { - sourceLoopId = loopId - } - - if (loopConfig.nodes.includes(target)) { - targetLoopId = loopId - } - } + // Find the innermost loop for each block. In nested loops a block appears + // in multiple loop configs; we need the most deeply nested one. + const sourceLoopId = this.findInnermostLoop(source, dag) + const targetLoopId = this.findInnermostLoop(target, dag) return sourceLoopId !== targetLoopId } + /** + * Finds the innermost loop containing a block. When a block is in nested + * loops (A contains B, both list the block), returns B (the one that + * doesn't contain any other candidate loop). + */ + private findInnermostLoop(blockId: string, dag: DAG): string | undefined { + const candidates: string[] = [] + for (const [loopId, loopConfig] of dag.loopConfigs) { + if (loopConfig.nodes.includes(blockId)) { + candidates.push(loopId) + } + } + if (candidates.length <= 1) return candidates[0] + + return candidates.find((candidateId) => + candidates.every((otherId) => { + if (otherId === candidateId) return true + const candidateConfig = dag.loopConfigs.get(candidateId) + return !candidateConfig?.nodes.includes(otherId) + }) + ) + } + private isEdgeReachable( source: string, target: string, @@ -406,24 +479,75 @@ export class EdgeConstructor { this.addEdge(dag, sourceNodeId, targetNodeId, sourceHandle, targetHandle) } + /** + * Resolves the DAG node to inspect for a given loop child. + * If the child is a nested subflow (loop or parallel), returns its sentinel node; + * otherwise returns the regular DAG node. + */ + private resolveLoopChildNode( + nodeId: string, + dag: DAG, + sentinel: 'start' | 'end' + ): { resolvedId: string; node: DAGNode | undefined } { + if (dag.loopConfigs.has(nodeId)) { + const resolvedId = + sentinel === 'start' ? buildSentinelStartId(nodeId) : buildSentinelEndId(nodeId) + return { resolvedId, node: dag.nodes.get(resolvedId) } + } + if (dag.parallelConfigs.has(nodeId)) { + const resolvedId = + sentinel === 'start' + ? buildParallelSentinelStartId(nodeId) + : buildParallelSentinelEndId(nodeId) + return { resolvedId, node: dag.nodes.get(resolvedId) } + } + return { resolvedId: nodeId, node: dag.nodes.get(nodeId) } + } + + private resolveLoopBlockToSentinelStart(nodeId: string, dag: DAG): string { + return this.resolveLoopChildNode(nodeId, dag, 'start').resolvedId + } + + private resolveLoopBlockToSentinelEnd(nodeId: string, dag: DAG): string { + return this.resolveLoopChildNode(nodeId, dag, 'end').resolvedId + } + + /** + * Builds the set of effective DAG node IDs for a loop's children, + * mapping nested subflow block IDs (loops and parallels) to their sentinel IDs. + */ + private buildEffectiveNodeSet(nodes: string[], dag: DAG): Set { + const effective = new Set() + for (const nodeId of nodes) { + if (dag.loopConfigs.has(nodeId)) { + effective.add(buildSentinelStartId(nodeId)) + effective.add(buildSentinelEndId(nodeId)) + } else if (dag.parallelConfigs.has(nodeId)) { + effective.add(buildParallelSentinelStartId(nodeId)) + effective.add(buildParallelSentinelEndId(nodeId)) + } else { + effective.add(nodeId) + } + } + return effective + } + private findLoopBoundaryNodes( nodes: string[], - dag: DAG, - reachableBlocks: Set + dag: DAG ): { startNodes: string[]; terminalNodes: string[] } { - const nodesSet = new Set(nodes) + const effectiveNodeSet = this.buildEffectiveNodeSet(nodes, dag) const startNodesSet = new Set() const terminalNodesSet = new Set() for (const nodeId of nodes) { - const node = dag.nodes.get(nodeId) + const { node } = this.resolveLoopChildNode(nodeId, dag, 'start') if (!node) continue let hasIncomingFromLoop = false - for (const incomingNodeId of node.incomingEdges) { - if (nodesSet.has(incomingNodeId)) { + if (effectiveNodeSet.has(incomingNodeId)) { hasIncomingFromLoop = true break } @@ -435,14 +559,17 @@ export class EdgeConstructor { } for (const nodeId of nodes) { - const node = dag.nodes.get(nodeId) + const { node } = this.resolveLoopChildNode(nodeId, dag, 'end') if (!node) continue let hasOutgoingToLoop = false + for (const [, edge] of node.outgoingEdges) { + const isBackEdge = + edge.sourceHandle === EDGE.LOOP_CONTINUE || edge.sourceHandle === EDGE.LOOP_CONTINUE_ALT + if (isBackEdge) continue - for (const [_, edge] of node.outgoingEdges) { - if (nodesSet.has(edge.target)) { + if (effectiveNodeSet.has(edge.target)) { hasOutgoingToLoop = true break } @@ -468,39 +595,77 @@ export class EdgeConstructor { const terminalNodes: string[] = [] for (const nodeId of nodes) { - const templateId = buildBranchNodeId(nodeId, 0) - const templateNode = dag.nodes.get(templateId) + // For nested subflow containers, use their sentinel nodes for boundary detection + const { startNode, endNode } = this.resolveParallelChildNodes(nodeId, dag) - if (!templateNode) continue + if (!startNode && !endNode) continue - let hasIncomingFromParallel = false - for (const incomingNodeId of templateNode.incomingEdges) { - const originalNodeId = extractBaseBlockId(incomingNodeId) - if (nodesSet.has(originalNodeId)) { - hasIncomingFromParallel = true - break + // Entry detection: check if the start-facing node has incoming edges from within the parallel + if (startNode) { + let hasIncomingFromParallel = false + for (const incomingNodeId of startNode.incomingEdges) { + const originalNodeId = normalizeNodeId(incomingNodeId) + if (nodesSet.has(originalNodeId)) { + hasIncomingFromParallel = true + break + } + } + if (!hasIncomingFromParallel) { + entryNodes.push(nodeId) } } - if (!hasIncomingFromParallel) { - entryNodes.push(nodeId) - } - let hasOutgoingToParallel = false - for (const [, edge] of templateNode.outgoingEdges) { - const originalTargetId = extractBaseBlockId(edge.target) - if (nodesSet.has(originalTargetId)) { - hasOutgoingToParallel = true - break + // Terminal detection: check if the end-facing node has outgoing edges to within the parallel + if (endNode) { + let hasOutgoingToParallel = false + for (const [, edge] of endNode.outgoingEdges) { + // Skip loop back-edges — they don't count as forward edges within the parallel + const isBackEdge = + edge.sourceHandle === EDGE.LOOP_CONTINUE || edge.sourceHandle === EDGE.LOOP_CONTINUE_ALT + if (isBackEdge) continue + + const originalTargetId = normalizeNodeId(edge.target) + if (nodesSet.has(originalTargetId)) { + hasOutgoingToParallel = true + break + } + } + if (!hasOutgoingToParallel) { + terminalNodes.push(nodeId) } - } - if (!hasOutgoingToParallel) { - terminalNodes.push(nodeId) } } return { entryNodes, terminalNodes } } + /** + * Resolves a child node inside a parallel to the correct DAG nodes for boundary detection. + * For regular blocks, returns the branch template node for both start and end. + * For nested parallels, returns the inner parallel's sentinel-start and sentinel-end. + * For nested loops, returns the inner loop's sentinel-start and sentinel-end. + */ + private resolveParallelChildNodes( + nodeId: string, + dag: DAG + ): { startNode: DAGNode | undefined; endNode: DAGNode | undefined } { + if (dag.parallelConfigs.has(nodeId)) { + return { + startNode: dag.nodes.get(buildParallelSentinelStartId(nodeId)), + endNode: dag.nodes.get(buildParallelSentinelEndId(nodeId)), + } + } + if (dag.loopConfigs.has(nodeId)) { + return { + startNode: dag.nodes.get(buildSentinelStartId(nodeId)), + endNode: dag.nodes.get(buildSentinelEndId(nodeId)), + } + } + // Regular block — use branch template node for both + const templateNode = dag.nodes.get(buildBranchNodeId(nodeId, 0)) + return { startNode: templateNode, endNode: templateNode } + } + private getParallelId(blockId: string, dag: DAG): string | null { for (const [parallelId, parallelConfig] of dag.parallelConfigs) { if (parallelConfig.nodes.includes(blockId)) { diff --git a/apps/sim/executor/execution/block-executor.ts b/apps/sim/executor/execution/block-executor.ts index 9325aa2861..46b2e554d2 100644 --- a/apps/sim/executor/execution/block-executor.ts +++ b/apps/sim/executor/execution/block-executor.ts @@ -20,7 +20,7 @@ import { ChildWorkflowError } from '@/executor/errors/child-workflow-error' import type { BlockStateWriter, ContextExtensions, - IterationContext, + WorkflowNodeMetadata, } from '@/executor/execution/types' import { generatePauseContextId, @@ -36,11 +36,14 @@ import { } from '@/executor/types' import { streamingResponseFormatProcessor } from '@/executor/utils' import { buildBlockExecutionError, normalizeError } from '@/executor/utils/errors' +import { + buildUnifiedParentIterations, + getIterationContext, +} from '@/executor/utils/iteration-context' import { isJSONString } from '@/executor/utils/json' import { filterOutputForLog } from '@/executor/utils/output-filter' import type { VariableResolver } from '@/executor/variables/resolver' import type { SerializedBlock } from '@/serializer/types' -import type { SubflowType } from '@/stores/workflows/workflow/types' import { SYSTEM_SUBBLOCK_IDS } from '@/triggers/constants' const logger = createLogger('BlockExecutor') @@ -169,9 +172,10 @@ export class BlockExecutor { this.state.setBlockOutput(node.id, normalizedOutput, duration) if (!isSentinel && blockLog) { - const childWorkflowInstanceId = normalizedOutput._childWorkflowInstanceId as - | string - | undefined + const childWorkflowInstanceId = + typeof normalizedOutput._childWorkflowInstanceId === 'string' + ? normalizedOutput._childWorkflowInstanceId + : undefined const displayOutput = filterOutputForLog(block.metadata?.id || '', normalizedOutput, { block, }) @@ -205,15 +209,7 @@ export class BlockExecutor { } } - private buildNodeMetadata(node: DAGNode): { - nodeId: string - loopId?: string - parallelId?: string - branchIndex?: number - branchTotal?: number - originalBlockId?: string - isLoopNode?: boolean - } { + private buildNodeMetadata(node: DAGNode): WorkflowNodeMetadata { const metadata = node?.metadata ?? {} return { nodeId: node.id, @@ -367,6 +363,11 @@ export class BlockExecutor { } } + const containerId = parallelId ?? loopId + const parentIterations = containerId + ? buildUnifiedParentIterations(ctx, containerId) + : undefined + return { blockId, blockName, @@ -379,6 +380,7 @@ export class BlockExecutor { loopId, parallelId, iterationIndex, + ...(parentIterations?.length && { parentIterations }), } } @@ -447,7 +449,7 @@ export class BlockExecutor { const blockName = block.metadata?.name ?? blockId const blockType = block.metadata?.id ?? DEFAULTS.BLOCK_TYPE - const iterationContext = this.getIterationContext(ctx, node) + const iterationContext = getIterationContext(ctx, node?.metadata) if (this.contextExtensions.onBlockStart) { this.contextExtensions.onBlockStart( @@ -477,7 +479,7 @@ export class BlockExecutor { const blockName = block.metadata?.name ?? blockId const blockType = block.metadata?.id ?? DEFAULTS.BLOCK_TYPE - const iterationContext = this.getIterationContext(ctx, node) + const iterationContext = getIterationContext(ctx, node?.metadata) if (this.contextExtensions.onBlockComplete) { this.contextExtensions.onBlockComplete( @@ -499,47 +501,6 @@ export class BlockExecutor { } } - private createIterationContext( - iterationCurrent: number, - iterationType: SubflowType, - iterationContainerId?: string, - iterationTotal?: number - ): IterationContext { - return { - iterationCurrent, - iterationTotal, - iterationType, - iterationContainerId, - } - } - - private getIterationContext(ctx: ExecutionContext, node: DAGNode): IterationContext | undefined { - if (!node?.metadata) return undefined - - if (node.metadata.branchIndex !== undefined && node.metadata.branchTotal !== undefined) { - return this.createIterationContext( - node.metadata.branchIndex, - 'parallel', - node.metadata.parallelId, - node.metadata.branchTotal - ) - } - - if (node.metadata.isLoopNode && node.metadata.loopId) { - const loopScope = ctx.loopExecutions?.get(node.metadata.loopId) - if (loopScope && loopScope.iteration !== undefined) { - return this.createIterationContext( - loopScope.iteration, - 'loop', - node.metadata.loopId, - loopScope.maxIterations - ) - } - } - - return undefined - } - private preparePauseResumeSelfReference( ctx: ExecutionContext, node: DAGNode, diff --git a/apps/sim/executor/execution/executor.ts b/apps/sim/executor/execution/executor.ts index a888409347..67d3b4c24b 100644 --- a/apps/sim/executor/execution/executor.ts +++ b/apps/sim/executor/execution/executor.ts @@ -1,5 +1,6 @@ import { createLogger } from '@sim/logger' import { StartBlockPath } from '@/lib/workflows/triggers/triggers' +import type { DAG } from '@/executor/dag/builder' import { DAGBuilder } from '@/executor/dag/builder' import { BlockExecutor } from '@/executor/execution/block-executor' import { EdgeManager } from '@/executor/execution/edge-manager' @@ -32,6 +33,7 @@ import { } from '@/executor/utils/subflow-utils' import { VariableResolver } from '@/executor/variables/resolver' import type { SerializedWorkflow } from '@/serializer/types' +import type { SubflowType } from '@/stores/workflows/workflow/types' const logger = createLogger('DAGExecutor') @@ -67,25 +69,9 @@ export class DAGExecutor { savedIncomingEdges, }) const { context, state } = this.createExecutionContext(workflowId, triggerBlockId) + context.subflowParentMap = this.buildSubflowParentMap(dag) - const resolver = new VariableResolver(this.workflow, this.workflowVariables, state) - const loopOrchestrator = new LoopOrchestrator(dag, state, resolver) - loopOrchestrator.setContextExtensions(this.contextExtensions) - const parallelOrchestrator = new ParallelOrchestrator(dag, state) - parallelOrchestrator.setResolver(resolver) - parallelOrchestrator.setContextExtensions(this.contextExtensions) - const allHandlers = createBlockHandlers() - const blockExecutor = new BlockExecutor(allHandlers, resolver, this.contextExtensions, state) - const edgeManager = new EdgeManager(dag) - loopOrchestrator.setEdgeManager(edgeManager) - const nodeOrchestrator = new NodeExecutionOrchestrator( - dag, - state, - blockExecutor, - loopOrchestrator, - parallelOrchestrator - ) - const engine = new ExecutionEngine(context, dag, edgeManager, nodeOrchestrator) + const engine = this.buildExecutionPipeline(context, dag, state) return await engine.run(triggerBlockId) } @@ -208,17 +194,30 @@ export class DAGExecutor { snapshotState: filteredSnapshot, runFromBlockContext, }) + context.subflowParentMap = this.buildSubflowParentMap(dag) + const engine = this.buildExecutionPipeline(context, dag, state) + return await engine.run() + } + + private buildExecutionPipeline(context: ExecutionContext, dag: DAG, state: ExecutionState) { const resolver = new VariableResolver(this.workflow, this.workflowVariables, state) - const loopOrchestrator = new LoopOrchestrator(dag, state, resolver) - loopOrchestrator.setContextExtensions(this.contextExtensions) - const parallelOrchestrator = new ParallelOrchestrator(dag, state) - parallelOrchestrator.setResolver(resolver) - parallelOrchestrator.setContextExtensions(this.contextExtensions) const allHandlers = createBlockHandlers() const blockExecutor = new BlockExecutor(allHandlers, resolver, this.contextExtensions, state) const edgeManager = new EdgeManager(dag) - loopOrchestrator.setEdgeManager(edgeManager) + const loopOrchestrator = new LoopOrchestrator( + dag, + state, + resolver, + this.contextExtensions, + edgeManager + ) + const parallelOrchestrator = new ParallelOrchestrator( + dag, + state, + resolver, + this.contextExtensions + ) const nodeOrchestrator = new NodeExecutionOrchestrator( dag, state, @@ -226,9 +225,7 @@ export class DAGExecutor { loopOrchestrator, parallelOrchestrator ) - const engine = new ExecutionEngine(context, dag, edgeManager, nodeOrchestrator) - - return await engine.run() + return new ExecutionEngine(context, dag, edgeManager, nodeOrchestrator) } private createExecutionContext( @@ -371,6 +368,37 @@ export class DAGExecutor { return { context, state } } + /** + * Builds a unified child-subflow → parent-subflow mapping that covers all nesting + * combinations: loop-in-loop, parallel-in-parallel, loop-in-parallel, parallel-in-loop. + * Used by the iteration context builder to walk the full ancestor chain for SSE events. + */ + private buildSubflowParentMap( + dag: DAG + ): Map { + const parentMap = new Map() + + // Scan loop configs: children can be loops or parallels + for (const [loopId, config] of dag.loopConfigs) { + for (const nodeId of config.nodes) { + if (dag.loopConfigs.has(nodeId) || dag.parallelConfigs.has(nodeId)) { + parentMap.set(nodeId, { parentId: loopId, parentType: 'loop' }) + } + } + } + + // Scan parallel configs: children can be parallels or loops + for (const [parallelId, config] of dag.parallelConfigs) { + for (const nodeId of config.nodes ?? []) { + if (dag.parallelConfigs.has(nodeId) || dag.loopConfigs.has(nodeId)) { + parentMap.set(nodeId, { parentId: parallelId, parentType: 'parallel' }) + } + } + } + + return parentMap + } + private initializeStarterBlock( context: ExecutionContext, state: ExecutionState, diff --git a/apps/sim/executor/execution/types.ts b/apps/sim/executor/execution/types.ts index bea082fe8a..933aaa2060 100644 --- a/apps/sim/executor/execution/types.ts +++ b/apps/sim/executor/execution/types.ts @@ -1,5 +1,11 @@ import type { Edge } from 'reactflow' -import type { BlockLog, BlockState, NormalizedBlockOutput } from '@/executor/types' +import type { NodeMetadata } from '@/executor/dag/types' +import type { + BlockLog, + BlockState, + NormalizedBlockOutput, + StreamingExecution, +} from '@/executor/types' import type { RunFromBlockContext } from '@/executor/utils/run-from-block' import type { SubflowType } from '@/stores/workflows/workflow/types' @@ -49,11 +55,45 @@ export interface SerializableExecutionState { completedPauseContexts?: string[] } +/** + * Represents the iteration state of an ancestor subflow in a nested chain. + * Used to propagate parent iteration context through SSE events for both + * loop-in-loop and parallel-in-parallel nesting hierarchies. + */ +export interface ParentIteration { + iterationCurrent: number + iterationTotal?: number + iterationType: SubflowType + iterationContainerId: string +} + export interface IterationContext { iterationCurrent: number iterationTotal?: number iterationType: SubflowType + /** + * Block ID of the loop or parallel container owning this iteration. + * Optional because generic `` references may resolve before + * the container ID is known (e.g., via `context.loopScope` fallback). + * Always present on {@link ParentIteration} entries since those are built + * from fully resolved ancestor loops. + */ iterationContainerId?: string + parentIterations?: ParentIteration[] +} + +/** + * Metadata passed to block handlers that execute within subflow contexts + * (loops, parallels, child workflows). Extends the DAG node metadata with + * runtime identifiers needed for execution tracking. + */ +export interface WorkflowNodeMetadata + extends Pick< + NodeMetadata, + 'loopId' | 'parallelId' | 'branchIndex' | 'branchTotal' | 'originalBlockId' | 'isLoopNode' + > { + nodeId: string + executionOrder?: number } export interface ChildWorkflowContext { @@ -68,7 +108,7 @@ export interface ChildWorkflowContext { } export interface ExecutionCallbacks { - onStream?: (streamingExec: any) => Promise + onStream?: (streamingExec: StreamingExecution) => Promise onBlockStart?: ( blockId: string, blockName: string, @@ -122,7 +162,7 @@ export interface ContextExtensions { abortSignal?: AbortSignal includeFileBase64?: boolean base64MaxBytes?: number - onStream?: (streamingExecution: unknown) => Promise + onStream?: (streamingExecution: StreamingExecution) => Promise onBlockStart?: ( blockId: string, blockName: string, diff --git a/apps/sim/executor/handlers/agent/agent-handler.test.ts b/apps/sim/executor/handlers/agent/agent-handler.test.ts index 98560156e4..87fba03a41 100644 --- a/apps/sim/executor/handlers/agent/agent-handler.test.ts +++ b/apps/sim/executor/handlers/agent/agent-handler.test.ts @@ -186,7 +186,7 @@ describe('AgentBlockHandler', () => { }) }) - mockTransformBlockTool.mockImplementation((tool: any) => ({ + mockTransformBlockTool.mockImplementation((tool: { id?: string; operation?: string }) => ({ id: `transformed_${tool.id}`, name: `${tool.id}_${tool.operation}`, description: 'Transformed tool', @@ -341,9 +341,15 @@ describe('AgentBlockHandler', () => { expect(tools.length).toBe(2) - const autoTool = tools.find((t: any) => t.name === 'auto_tool') - const forceTool = tools.find((t: any) => t.name === 'force_tool') - const noneTool = tools.find((t: any) => t.name === 'none_tool') + const autoTool = tools.find( + (t: { name?: string; id?: string; usageControl?: string }) => t.name === 'auto_tool' + ) + const forceTool = tools.find( + (t: { name?: string; id?: string; usageControl?: string }) => t.name === 'force_tool' + ) + const noneTool = tools.find( + (t: { name?: string; id?: string; usageControl?: string }) => t.name === 'none_tool' + ) expect(autoTool).toBeDefined() expect(forceTool).toBeDefined() @@ -392,7 +398,9 @@ describe('AgentBlockHandler', () => { expect(requestBody.tools.length).toBe(2) - const toolIds = requestBody.tools.map((t: any) => t.id) + const toolIds = requestBody.tools.map( + (t: { name?: string; id?: string; usageControl?: string }) => t.id + ) expect(toolIds).toContain('transformed_tool_1') expect(toolIds).toContain('transformed_tool_3') expect(toolIds).not.toContain('transformed_tool_2') @@ -421,7 +429,7 @@ describe('AgentBlockHandler', () => { ], } - mockTransformBlockTool.mockImplementation((tool: any) => ({ + mockTransformBlockTool.mockImplementation((tool: { id?: string; operation?: string }) => ({ id: `transformed_${tool.id}`, name: `${tool.id}_${tool.operation}`, description: 'Transformed tool', @@ -502,13 +510,19 @@ describe('AgentBlockHandler', () => { expect(requestBody.tools.length).toBe(2) - const toolNames = requestBody.tools.map((t: any) => t.name) + const toolNames = requestBody.tools.map( + (t: { name?: string; id?: string; usageControl?: string }) => t.name + ) expect(toolNames).toContain('custom_tool_auto') expect(toolNames).toContain('custom_tool_force') expect(toolNames).not.toContain('custom_tool_none') - const autoTool = requestBody.tools.find((t: any) => t.name === 'custom_tool_auto') - const forceTool = requestBody.tools.find((t: any) => t.name === 'custom_tool_force') + const autoTool = requestBody.tools.find( + (t: { name?: string; id?: string; usageControl?: string }) => t.name === 'custom_tool_auto' + ) + const forceTool = requestBody.tools.find( + (t: { name?: string; id?: string; usageControl?: string }) => t.name === 'custom_tool_force' + ) expect(autoTool.usageControl).toBe('auto') expect(forceTool.usageControl).toBe('force') @@ -1473,7 +1487,7 @@ describe('AgentBlockHandler', () => { timing: { total: 200 }, }) - mockTransformBlockTool.mockImplementation((tool: any) => ({ + mockTransformBlockTool.mockImplementation((tool: { id?: string; operation?: string }) => ({ id: tool.schema?.function?.name || `mcp-${tool.title.toLowerCase().replace(' ', '-')}`, name: tool.schema?.function?.name || tool.title, description: tool.schema?.function?.description || `MCP tool: ${tool.title}`, diff --git a/apps/sim/executor/handlers/agent/agent-handler.ts b/apps/sim/executor/handlers/agent/agent-handler.ts index 3c103cc942..5ca02a22e1 100644 --- a/apps/sim/executor/handlers/agent/agent-handler.ts +++ b/apps/sim/executor/handlers/agent/agent-handler.ts @@ -591,12 +591,15 @@ export class AgentBlockHandler implements BlockHandler { serverId: string toolName: string description: string - schema: any - userProvidedParams: Record - usageControl?: string - }): Promise { + schema: Record + userProvidedParams: Record + usageControl?: 'auto' | 'force' | 'none' + }) { const { filterSchemaForLLM } = await import('@/tools/params') - const filteredSchema = filterSchemaForLLM(config.schema, config.userProvidedParams) + const filteredSchema = filterSchemaForLLM( + config.schema as unknown as Parameters[0], + config.userProvidedParams as Record + ) const toolId = createMcpToolId(config.serverId, config.toolName) return { diff --git a/apps/sim/executor/handlers/workflow/workflow-handler.ts b/apps/sim/executor/handlers/workflow/workflow-handler.ts index c5ad12e62e..4821d86029 100644 --- a/apps/sim/executor/handlers/workflow/workflow-handler.ts +++ b/apps/sim/executor/handlers/workflow/workflow-handler.ts @@ -7,7 +7,7 @@ import type { BlockOutput } from '@/blocks/types' import { Executor } from '@/executor' import { BlockType, DEFAULTS, HTTP } from '@/executor/constants' import { ChildWorkflowError } from '@/executor/errors/child-workflow-error' -import type { IterationContext } from '@/executor/execution/types' +import type { WorkflowNodeMetadata } from '@/executor/execution/types' import type { BlockHandler, ExecutionContext, @@ -16,6 +16,7 @@ import type { } from '@/executor/types' import { hasExecutionResult } from '@/executor/utils/errors' import { buildAPIUrl, buildAuthHeaders } from '@/executor/utils/http' +import { getIterationContext } from '@/executor/utils/iteration-context' import { parseJSON } from '@/executor/utils/json' import { lazyCleanupInputMapping } from '@/executor/utils/lazy-cleanup' import { Serializer } from '@/serializer' @@ -47,41 +48,23 @@ export class WorkflowBlockHandler implements BlockHandler { block: SerializedBlock, inputs: Record ): Promise { - return this._executeCore(ctx, block, inputs) + return this.executeCore(ctx, block, inputs) } async executeWithNode( ctx: ExecutionContext, block: SerializedBlock, inputs: Record, - nodeMetadata: { - nodeId: string - loopId?: string - parallelId?: string - branchIndex?: number - branchTotal?: number - originalBlockId?: string - isLoopNode?: boolean - executionOrder?: number - } + nodeMetadata: WorkflowNodeMetadata ): Promise { - return this._executeCore(ctx, block, inputs, nodeMetadata) + return this.executeCore(ctx, block, inputs, nodeMetadata) } - private async _executeCore( + private async executeCore( ctx: ExecutionContext, block: SerializedBlock, inputs: Record, - nodeMetadata?: { - nodeId: string - loopId?: string - parallelId?: string - branchIndex?: number - branchTotal?: number - originalBlockId?: string - isLoopNode?: boolean - executionOrder?: number - } + nodeMetadata?: WorkflowNodeMetadata ): Promise { logger.info(`Executing workflow block: ${block.id}`) @@ -164,13 +147,19 @@ export class WorkflowBlockHandler implements BlockHandler { const childDepth = (ctx.childWorkflowContext?.depth ?? 0) + 1 const shouldPropagateCallbacks = childDepth <= DEFAULTS.MAX_SSE_CHILD_DEPTH + if (!shouldPropagateCallbacks) { + logger.info('Dropping SSE callbacks beyond max child depth', { + childDepth, + maxDepth: DEFAULTS.MAX_SSE_CHILD_DEPTH, + childWorkflowName, + }) + } + if (shouldPropagateCallbacks) { const effectiveBlockId = nodeMetadata ? (nodeMetadata.originalBlockId ?? nodeMetadata.nodeId) : block.id - const iterationContext = nodeMetadata - ? this.getIterationContext(ctx, nodeMetadata) - : undefined + const iterationContext = nodeMetadata ? getIterationContext(ctx, nodeMetadata) : undefined ctx.onChildWorkflowInstanceReady?.( effectiveBlockId, instanceId, @@ -196,7 +185,7 @@ export class WorkflowBlockHandler implements BlockHandler { ...(shouldPropagateCallbacks && { onBlockStart: ctx.onBlockStart, onBlockComplete: ctx.onBlockComplete, - onStream: ctx.onStream as ((streamingExecution: unknown) => Promise) | undefined, + onStream: ctx.onStream, onChildWorkflowInstanceReady: ctx.onChildWorkflowInstanceReady, childWorkflowContext: { parentBlockId: instanceId, @@ -268,40 +257,6 @@ export class WorkflowBlockHandler implements BlockHandler { } } - private getIterationContext( - ctx: ExecutionContext, - nodeMetadata: { - loopId?: string - parallelId?: string - branchIndex?: number - branchTotal?: number - isLoopNode?: boolean - } - ): IterationContext | undefined { - if (nodeMetadata.branchIndex !== undefined && nodeMetadata.branchTotal !== undefined) { - return { - iterationCurrent: nodeMetadata.branchIndex, - iterationTotal: nodeMetadata.branchTotal, - iterationType: 'parallel', - iterationContainerId: nodeMetadata.parallelId, - } - } - - if (nodeMetadata.isLoopNode && nodeMetadata.loopId) { - const loopScope = ctx.loopExecutions?.get(nodeMetadata.loopId) - if (loopScope && loopScope.iteration !== undefined) { - return { - iterationCurrent: loopScope.iteration, - iterationTotal: loopScope.maxIterations, - iterationType: 'loop', - iterationContainerId: nodeMetadata.loopId, - } - } - } - - return undefined - } - /** * Builds a cleaner error message for nested workflow errors. * Parses nested error messages to extract workflow chain and root error. @@ -627,6 +582,6 @@ export class WorkflowBlockHandler implements BlockHandler { result, childTraceSpans: childTraceSpans || [], _childWorkflowInstanceId: instanceId, - } as Record + } as unknown as BlockOutput } } diff --git a/apps/sim/executor/human-in-the-loop/utils.ts b/apps/sim/executor/human-in-the-loop/utils.ts index 1b060cf833..2d14788a63 100644 --- a/apps/sim/executor/human-in-the-loop/utils.ts +++ b/apps/sim/executor/human-in-the-loop/utils.ts @@ -1,12 +1,10 @@ import { PARALLEL } from '@/executor/constants' +import type { NodeMetadata } from '@/executor/dag/types' import type { ExecutionContext, LoopPauseScope, ParallelPauseScope } from '@/executor/types' -interface NodeMetadataLike { +interface NodeMetadataLike + extends Pick { nodeId: string - loopId?: string - parallelId?: string - branchIndex?: number - branchTotal?: number } export function generatePauseContextId( diff --git a/apps/sim/executor/orchestrators/loop.ts b/apps/sim/executor/orchestrators/loop.ts index 456838d1ee..582d7feb77 100644 --- a/apps/sim/executor/orchestrators/loop.ts +++ b/apps/sim/executor/orchestrators/loop.ts @@ -2,7 +2,7 @@ import { createLogger } from '@sim/logger' import { generateRequestId } from '@/lib/core/utils/request' import { isExecutionCancelled, isRedisCancellationEnabled } from '@/lib/execution/cancellation' import { executeInIsolatedVM } from '@/lib/execution/isolated-vm' -import { buildLoopIndexCondition, DEFAULTS, EDGE } from '@/executor/constants' +import { buildLoopIndexCondition, DEFAULTS, EDGE, PARALLEL } from '@/executor/constants' import type { DAG } from '@/executor/dag/builder' import type { EdgeManager } from '@/executor/execution/edge-manager' import type { LoopScope } from '@/executor/execution/state' @@ -13,9 +13,12 @@ import { type NormalizedBlockOutput, } from '@/executor/types' import type { LoopConfigWithNodes } from '@/executor/types/loop' +import { buildContainerIterationContext } from '@/executor/utils/iteration-context' import { replaceValidReferences } from '@/executor/utils/reference-validation' import { addSubflowErrorLog, + buildParallelSentinelEndId, + buildParallelSentinelStartId, buildSentinelEndId, buildSentinelStartId, extractBaseBlockId, @@ -39,23 +42,14 @@ export interface LoopContinuationResult { } export class LoopOrchestrator { - private edgeManager: EdgeManager | null = null - private contextExtensions: ContextExtensions | null = null - constructor( private dag: DAG, private state: BlockStateController, - private resolver: VariableResolver + private resolver: VariableResolver, + private contextExtensions: ContextExtensions | null = null, + private edgeManager: EdgeManager | null = null ) {} - setContextExtensions(contextExtensions: ContextExtensions): void { - this.contextExtensions = contextExtensions - } - - setEdgeManager(edgeManager: EdgeManager): void { - this.edgeManager = edgeManager - } - initializeLoopScope(ctx: ExecutionContext, loopId: string): LoopScope { const loopConfig = this.dag.loopConfigs.get(loopId) as SerializedLoop | undefined if (!loopConfig) { @@ -100,7 +94,7 @@ export class LoopOrchestrator { scope.loopType = 'forEach' let items: any[] try { - items = this.resolveForEachItems(ctx, loopConfig.forEachItems) + items = resolveArrayInput(ctx, loopConfig.forEachItems, this.resolver) } catch (error) { const errorMessage = `ForEach loop resolution failed: ${error instanceof Error ? error.message : String(error)}` logger.error(errorMessage, { loopId, forEachItems: loopConfig.forEachItems }) @@ -283,16 +277,23 @@ export class LoopOrchestrator { const output = { results } this.state.setBlockOutput(loopId, output, DEFAULTS.EXECUTION_TIME) - // Emit onBlockComplete for the loop container so the UI can track it if (this.contextExtensions?.onBlockComplete) { const now = new Date().toISOString() - this.contextExtensions.onBlockComplete(loopId, 'Loop', 'loop', { - output, - executionTime: DEFAULTS.EXECUTION_TIME, - startedAt: now, - executionOrder: getNextExecutionOrder(ctx), - endedAt: now, - }) + const iterationContext = buildContainerIterationContext(ctx, loopId) + + this.contextExtensions.onBlockComplete( + loopId, + 'Loop', + 'loop', + { + output, + executionTime: DEFAULTS.EXECUTION_TIME, + startedAt: now, + executionOrder: getNextExecutionOrder(ctx), + endedAt: now, + }, + iterationContext + ) } return { @@ -327,21 +328,211 @@ export class LoopOrchestrator { return result } - clearLoopExecutionState(loopId: string): void { - const loopConfig = this.dag.loopConfigs.get(loopId) as LoopConfigWithNodes | undefined - if (!loopConfig) { - logger.warn('Loop config not found for state clearing', { loopId }) - return + clearLoopExecutionState(loopId: string, ctx: ExecutionContext): void { + const allNodeIds = this.collectAllLoopNodeIds(loopId) + + for (const nodeId of allNodeIds) { + this.state.unmarkExecuted(nodeId) } + this.resetNestedLoopScopes(loopId, ctx) + this.resetNestedParallelScopes(loopId, ctx) + } + + /** + * Deletes loop scopes for any nested loops so they re-initialize + * on the next outer iteration. + */ + private resetNestedLoopScopes(loopId: string, ctx: ExecutionContext): void { + const loopConfig = this.dag.loopConfigs.get(loopId) as LoopConfigWithNodes | undefined + if (!loopConfig) return + + for (const nodeId of loopConfig.nodes) { + if (this.dag.loopConfigs.has(nodeId)) { + ctx.loopExecutions?.delete(nodeId) + // Delete cloned loop variants (__obranch-N and __clone*) but not original + // subflowParentMap entries which are needed for SSE iteration context. + if (ctx.loopExecutions) { + const obranchPrefix = `${nodeId}__obranch-` + const cloneSeqPrefix = `${nodeId}__clone` + for (const key of ctx.loopExecutions.keys()) { + if (key.startsWith(obranchPrefix) || key.startsWith(cloneSeqPrefix)) { + ctx.loopExecutions.delete(key) + ctx.subflowParentMap?.delete(key) + } + } + } + this.resetNestedLoopScopes(nodeId, ctx) + } + } + } + + /** + * Deletes parallel scopes for any nested parallels (including cloned + * subflows with `__obranch-N` suffixes) so they re-initialize on the + * next outer loop iteration. + */ + private resetNestedParallelScopes(loopId: string, ctx: ExecutionContext): void { + const loopConfig = this.dag.loopConfigs.get(loopId) as LoopConfigWithNodes | undefined + if (!loopConfig) return + + for (const nodeId of loopConfig.nodes) { + if (this.dag.parallelConfigs.has(nodeId)) { + this.deleteParallelScopeAndClones(nodeId, ctx) + } else if (this.dag.loopConfigs.has(nodeId)) { + this.resetNestedParallelScopes(nodeId, ctx) + } + } + } + + /** + * Deletes a parallel scope and any cloned variants (`__obranch-N`), + * recursively handling nested subflows within the parallel. + */ + private deleteParallelScopeAndClones(parallelId: string, ctx: ExecutionContext): void { + ctx.parallelExecutions?.delete(parallelId) + // Delete cloned scopes (__obranch-N and __clone*) but not original subflowParentMap entries + if (ctx.parallelExecutions) { + const obranchPrefix = `${parallelId}__obranch-` + const clonePrefix = `${parallelId}__clone` + for (const key of ctx.parallelExecutions.keys()) { + if (key.startsWith(obranchPrefix) || key.startsWith(clonePrefix)) { + ctx.parallelExecutions.delete(key) + ctx.subflowParentMap?.delete(key) + } + } + } + + const parallelConfig = this.dag.parallelConfigs.get(parallelId) + if (parallelConfig?.nodes) { + for (const nodeId of parallelConfig.nodes) { + if (this.dag.parallelConfigs.has(nodeId)) { + this.deleteParallelScopeAndClones(nodeId, ctx) + } else if (this.dag.loopConfigs.has(nodeId)) { + ctx.loopExecutions?.delete(nodeId) + // Also delete cloned loop scopes (__obranch-N and __clone*) created by expandParallel + if (ctx.loopExecutions) { + const obranchPrefix = `${nodeId}__obranch-` + const cloneSeqPrefix = `${nodeId}__clone` + for (const key of ctx.loopExecutions.keys()) { + if (key.startsWith(obranchPrefix) || key.startsWith(cloneSeqPrefix)) { + ctx.loopExecutions.delete(key) + ctx.subflowParentMap?.delete(key) + } + } + } + this.resetNestedParallelScopes(nodeId, ctx) + } + } + } + } + + /** + * Collects all effective DAG node IDs for a loop, recursively including + * sentinel IDs for any nested subflow blocks (loops and parallels). + */ + private collectAllLoopNodeIds(loopId: string, visited = new Set()): Set { + if (visited.has(loopId)) return new Set() + visited.add(loopId) + + const loopConfig = this.dag.loopConfigs.get(loopId) as LoopConfigWithNodes | undefined + if (!loopConfig) return new Set() + const sentinelStartId = buildSentinelStartId(loopId) const sentinelEndId = buildSentinelEndId(loopId) - const loopNodes = loopConfig.nodes + const result = new Set([sentinelStartId, sentinelEndId]) - this.state.unmarkExecuted(sentinelStartId) - this.state.unmarkExecuted(sentinelEndId) - for (const loopNodeId of loopNodes) { - this.state.unmarkExecuted(loopNodeId) + for (const nodeId of loopConfig.nodes) { + if (this.dag.loopConfigs.has(nodeId)) { + for (const id of this.collectAllLoopNodeIds(nodeId, visited)) { + result.add(id) + } + this.collectClonedSubflowNodes(nodeId, result, visited) + } else if (this.dag.parallelConfigs.has(nodeId)) { + for (const id of this.collectAllParallelNodeIds(nodeId, visited)) { + result.add(id) + } + this.collectClonedSubflowNodes(nodeId, result, visited) + } else { + result.add(nodeId) + } + } + + return result + } + + /** + * Collects all effective DAG node IDs for a parallel, including + * sentinel IDs and branch template nodes, recursively handling nested subflows. + */ + private collectAllParallelNodeIds(parallelId: string, visited = new Set()): Set { + if (visited.has(parallelId)) return new Set() + visited.add(parallelId) + + const parallelConfig = this.dag.parallelConfigs.get(parallelId) + if (!parallelConfig) return new Set() + + const sentinelStartId = buildParallelSentinelStartId(parallelId) + const sentinelEndId = buildParallelSentinelEndId(parallelId) + const result = new Set([sentinelStartId, sentinelEndId]) + + for (const nodeId of parallelConfig.nodes) { + if (this.dag.loopConfigs.has(nodeId)) { + for (const id of this.collectAllLoopNodeIds(nodeId, visited)) { + result.add(id) + } + this.collectClonedSubflowNodes(nodeId, result, visited) + } else if (this.dag.parallelConfigs.has(nodeId)) { + for (const id of this.collectAllParallelNodeIds(nodeId, visited)) { + result.add(id) + } + this.collectClonedSubflowNodes(nodeId, result, visited) + } else { + result.add(nodeId) + this.collectAllBranchNodes(nodeId, result) + } + } + + return result + } + + /** + * Collects all branch nodes for a given base block ID by scanning the DAG. + * This captures dynamically created branches (1, 2, ...) beyond the template (0). + */ + private collectAllBranchNodes(baseNodeId: string, result: Set): void { + const prefix = `${baseNodeId}${PARALLEL.BRANCH.PREFIX}` + for (const dagNodeId of this.dag.nodes.keys()) { + if (dagNodeId.startsWith(prefix)) { + result.add(dagNodeId) + } + } + } + + /** + * Collects all cloned subflow variants (e.g., loop-1__obranch-N) and their + * descendant nodes by scanning the DAG configs. + */ + private collectClonedSubflowNodes( + originalId: string, + result: Set, + visited: Set + ): void { + const obranchPrefix = `${originalId}__obranch-` + const clonePrefix = `${originalId}__clone` + for (const loopId of this.dag.loopConfigs.keys()) { + if (loopId.startsWith(obranchPrefix) || loopId.startsWith(clonePrefix)) { + for (const id of this.collectAllLoopNodeIds(loopId, visited)) { + result.add(id) + } + } + } + for (const parallelId of this.dag.parallelConfigs.keys()) { + if (parallelId.startsWith(obranchPrefix) || parallelId.startsWith(clonePrefix)) { + for (const id of this.collectAllParallelNodeIds(parallelId, visited)) { + result.add(id) + } + } } } @@ -352,10 +543,7 @@ export class LoopOrchestrator { return } - const sentinelStartId = buildSentinelStartId(loopId) - const sentinelEndId = buildSentinelEndId(loopId) - const loopNodes = loopConfig.nodes - const allLoopNodeIds = new Set([sentinelStartId, sentinelEndId, ...loopNodes]) + const allLoopNodeIds = this.collectAllLoopNodeIds(loopId) if (this.edgeManager) { this.edgeManager.clearDeactivatedEdgesForNodes(allLoopNodeIds) @@ -365,8 +553,9 @@ export class LoopOrchestrator { const nodeToRestore = this.dag.nodes.get(nodeId) if (!nodeToRestore) continue - for (const [potentialSourceId, potentialSourceNode] of this.dag.nodes) { - if (!allLoopNodeIds.has(potentialSourceId)) continue + for (const potentialSourceId of allLoopNodeIds) { + const potentialSourceNode = this.dag.nodes.get(potentialSourceId) + if (!potentialSourceNode) continue for (const [, edge] of potentialSourceNode.outgoingEdges) { if (edge.target === nodeId) { @@ -412,23 +601,19 @@ export class LoopOrchestrator { return true } - // for: skip if maxIterations is 0 if (scope.loopType === 'for') { if (scope.maxIterations === 0) { logger.info('For loop has 0 iterations, skipping loop body', { loopId }) - // Set empty output for the loop this.state.setBlockOutput(loopId, { results: [] }, DEFAULTS.EXECUTION_TIME) return false } return true } - // doWhile: always execute at least once if (scope.loopType === 'doWhile') { return true } - // while: check condition before first iteration if (scope.loopType === 'while') { if (!scope.condition) { logger.warn('No condition defined for while loop', { loopId }) @@ -448,20 +633,6 @@ export class LoopOrchestrator { return true } - shouldExecuteLoopNode(_ctx: ExecutionContext, _nodeId: string, _loopId: string): boolean { - return true - } - - private findLoopForNode(nodeId: string): string | undefined { - for (const [loopId, config] of this.dag.loopConfigs) { - const nodes = (config as any).nodes || [] - if (nodes.includes(nodeId)) { - return loopId - } - } - return undefined - } - private async evaluateWhileCondition( ctx: ExecutionContext, condition: string, @@ -480,10 +651,9 @@ export class LoopOrchestrator { const evaluatedCondition = replaceValidReferences(condition, (match) => { const resolved = this.resolver.resolveSingleReference(ctx, '', match, scope) - logger.info('Resolved variable reference in loop condition', { + logger.debug('Resolved variable reference in loop condition', { reference: match, resolvedValue: resolved, - resolvedType: typeof resolved, }) if (resolved !== undefined) { if (typeof resolved === 'boolean' || typeof resolved === 'number') { @@ -538,8 +708,4 @@ export class LoopOrchestrator { return false } } - - private resolveForEachItems(ctx: ExecutionContext, items: any): any[] { - return resolveArrayInput(ctx, items, this.resolver) - } } diff --git a/apps/sim/executor/orchestrators/node.ts b/apps/sim/executor/orchestrators/node.ts index 535693a82f..862f7c1a2e 100644 --- a/apps/sim/executor/orchestrators/node.ts +++ b/apps/sim/executor/orchestrators/node.ts @@ -56,14 +56,6 @@ export class NodeExecutionOrchestrator { this.loopOrchestrator.initializeLoopScope(ctx, loopId) } - if (loopId && !this.loopOrchestrator.shouldExecuteLoopNode(ctx, nodeId, loopId)) { - return { - nodeId, - output: {}, - isFinalOutput: false, - } - } - const parallelId = node.metadata.parallelId if (parallelId && !this.parallelOrchestrator.getParallelScope(ctx, parallelId)) { const parallelConfig = this.dag.parallelConfigs.get(parallelId) @@ -276,7 +268,7 @@ export class NodeExecutionOrchestrator { ) { const loopId = node.metadata.loopId if (loopId) { - this.loopOrchestrator.clearLoopExecutionState(loopId) + this.loopOrchestrator.clearLoopExecutionState(loopId, ctx) this.loopOrchestrator.restoreLoopEdges(loopId) } } diff --git a/apps/sim/executor/orchestrators/parallel.ts b/apps/sim/executor/orchestrators/parallel.ts index 6d7ea2dfe2..5eb0038790 100644 --- a/apps/sim/executor/orchestrators/parallel.ts +++ b/apps/sim/executor/orchestrators/parallel.ts @@ -9,6 +9,7 @@ import { type NormalizedBlockOutput, } from '@/executor/types' import type { ParallelConfigWithNodes } from '@/executor/types/parallel' +import { buildContainerIterationContext } from '@/executor/utils/iteration-context' import { ParallelExpander } from '@/executor/utils/parallel-expansion' import { addSubflowErrorLog, @@ -36,23 +37,15 @@ export interface ParallelAggregationResult { } export class ParallelOrchestrator { - private resolver: VariableResolver | null = null - private contextExtensions: ContextExtensions | null = null private expander = new ParallelExpander() constructor( private dag: DAG, - private state: BlockStateWriter + private state: BlockStateWriter, + private resolver: VariableResolver | null = null, + private contextExtensions: ContextExtensions | null = null ) {} - setResolver(resolver: VariableResolver): void { - this.resolver = resolver - } - - setContextExtensions(contextExtensions: ContextExtensions): void { - this.contextExtensions = contextExtensions - } - initializeParallelScope( ctx: ExecutionContext, parallelId: string, @@ -97,7 +90,6 @@ export class ParallelOrchestrator { throw new Error(branchError) } - // Handle empty distribution - skip parallel body if (isEmpty || branchCount === 0) { const scope: ParallelScope = { parallelId, @@ -114,7 +106,6 @@ export class ParallelOrchestrator { } ctx.parallelExecutions.set(parallelId, scope) - // Set empty output for the parallel this.state.setBlockOutput(parallelId, { results: [] }) logger.info('Parallel scope initialized with empty distribution, skipping body', { @@ -125,7 +116,56 @@ export class ParallelOrchestrator { return scope } - const { entryNodes } = this.expander.expandParallel(this.dag, parallelId, branchCount, items) + const { entryNodes, clonedSubflows } = this.expander.expandParallel( + this.dag, + parallelId, + branchCount, + items + ) + + // Register cloned subflows in the parent map so iteration context resolves correctly. + // Build a per-branch clone map so nested clones point to the cloned parent, not the original. + if (clonedSubflows.length > 0 && ctx.subflowParentMap) { + const branchCloneMaps = new Map>() + for (const clone of clonedSubflows) { + let map = branchCloneMaps.get(clone.outerBranchIndex) + if (!map) { + map = new Map() + branchCloneMaps.set(clone.outerBranchIndex, map) + } + map.set(clone.originalId, clone.clonedId) + } + + for (const clone of clonedSubflows) { + const originalEntry = ctx.subflowParentMap.get(clone.originalId) + if (originalEntry) { + const cloneMap = branchCloneMaps.get(clone.outerBranchIndex) + const clonedParentId = cloneMap?.get(originalEntry.parentId) + if (clonedParentId) { + // Parent was also cloned — this is the original (branch 0) inside the cloned parent + ctx.subflowParentMap.set(clone.clonedId, { + parentId: clonedParentId, + parentType: originalEntry.parentType, + branchIndex: 0, + }) + } else { + // Parent was not cloned — direct child of the expanding parallel + ctx.subflowParentMap.set(clone.clonedId, { + parentId: parallelId, + parentType: 'parallel', + branchIndex: clone.outerBranchIndex, + }) + } + } else { + // Not in parent map — direct child of the expanding parallel + ctx.subflowParentMap.set(clone.clonedId, { + parentId: parallelId, + parentType: 'parallel', + branchIndex: clone.outerBranchIndex, + }) + } + } + } const scope: ParallelScope = { parallelId, @@ -210,10 +250,6 @@ export class ParallelOrchestrator { } private resolveDistributionItems(ctx: ExecutionContext, config: SerializedParallel): any[] { - if (config.parallelType === 'count') { - return [] - } - if ( config.distribution === undefined || config.distribution === null || @@ -261,22 +297,35 @@ export class ParallelOrchestrator { const results: NormalizedBlockOutput[][] = [] for (let i = 0; i < scope.totalBranches; i++) { - const branchOutputs = scope.branchOutputs.get(i) || [] - results.push(branchOutputs) + const branchOutputs = scope.branchOutputs.get(i) + if (!branchOutputs) { + logger.warn('Missing branch output during parallel aggregation', { parallelId, branch: i }) + } + results.push(branchOutputs ?? []) } const output = { results } this.state.setBlockOutput(parallelId, output) - // Emit onBlockComplete for the parallel container so the UI can track it + // Emit onBlockComplete for the parallel container so the UI can track it. + // When this parallel is nested inside a parent subflow (parallel or loop), emit + // iteration context so the terminal can group this event under the parent container. if (this.contextExtensions?.onBlockComplete) { const now = new Date().toISOString() - this.contextExtensions.onBlockComplete(parallelId, 'Parallel', 'parallel', { - output, - executionTime: 0, - startedAt: now, - executionOrder: getNextExecutionOrder(ctx), - endedAt: now, - }) + const iterationContext = buildContainerIterationContext(ctx, parallelId) + + this.contextExtensions.onBlockComplete( + parallelId, + 'Parallel', + 'parallel', + { + output, + executionTime: 0, + startedAt: now, + executionOrder: getNextExecutionOrder(ctx), + endedAt: now, + }, + iterationContext + ) } return { diff --git a/apps/sim/executor/types.ts b/apps/sim/executor/types.ts index bf672dfd94..585d6a2f58 100644 --- a/apps/sim/executor/types.ts +++ b/apps/sim/executor/types.ts @@ -4,10 +4,12 @@ import type { BlockOutput } from '@/blocks/types' import type { ChildWorkflowContext, IterationContext, + ParentIteration, SerializableExecutionState, } from '@/executor/execution/types' import type { RunFromBlockContext } from '@/executor/utils/run-from-block' import type { SerializedBlock, SerializedWorkflow } from '@/serializer/types' +import type { SubflowType } from '@/stores/workflows/workflow/types' export interface UserFile { id: string @@ -121,6 +123,8 @@ export interface BlockLog { loopId?: string parallelId?: string iterationIndex?: number + /** Full ancestor iteration chain for nested subflows (outermost → innermost). */ + parentIterations?: ParentIteration[] /** * Monotonically increasing integer (1, 2, 3, ...) for accurate block ordering. * Generated via getNextExecutionOrder() to ensure deterministic sorting. @@ -192,6 +196,17 @@ export interface ExecutionContext { completedLoops: Set + /** + * Unified parent map for subflow nesting (loop-in-loop, parallel-in-parallel, + * loop-in-parallel, parallel-in-loop). Maps any child subflow ID to its parent + * subflow ID and type, enabling the iteration context builder to walk the full + * ancestor chain regardless of subflow type. + */ + subflowParentMap?: Map< + string, + { parentId: string; parentType: SubflowType; branchIndex?: number } + > + loopExecutions?: Map< string, { diff --git a/apps/sim/executor/utils/builder-data.ts b/apps/sim/executor/utils/builder-data.ts index da471e5f4a..025aa0e577 100644 --- a/apps/sim/executor/utils/builder-data.ts +++ b/apps/sim/executor/utils/builder-data.ts @@ -4,19 +4,19 @@ export interface JSONProperty { id: string name: string type: string - value: any + value: unknown collapsed?: boolean } /** * Converts builder data (structured JSON properties) into a plain JSON object. */ -export function convertBuilderDataToJson(builderData: JSONProperty[]): any { +export function convertBuilderDataToJson(builderData: JSONProperty[]): Record { if (!Array.isArray(builderData)) { return {} } - const result: any = {} + const result: Record = {} for (const prop of builderData) { if (!prop.name || !prop.name.trim()) { @@ -38,7 +38,7 @@ export function convertBuilderDataToJsonString(builderData: JSONProperty[]): str return '{\n \n}' } - const result: any = {} + const result: Record = {} for (const prop of builderData) { if (!prop.name || !prop.name.trim()) { @@ -55,7 +55,7 @@ export function convertBuilderDataToJsonString(builderData: JSONProperty[]): str return jsonString } -export function convertPropertyValue(prop: JSONProperty): any { +export function convertPropertyValue(prop: JSONProperty): unknown { switch (prop.type) { case 'object': return convertObjectValue(prop.value) @@ -72,9 +72,9 @@ export function convertPropertyValue(prop: JSONProperty): any { } } -function convertObjectValue(value: any): any { +function convertObjectValue(value: unknown): unknown { if (Array.isArray(value)) { - return convertBuilderDataToJson(value) + return convertBuilderDataToJson(value as JSONProperty[]) } if (typeof value === 'string' && !isVariableReference(value)) { @@ -84,9 +84,9 @@ function convertObjectValue(value: any): any { return value } -function convertArrayValue(value: any): any { +function convertArrayValue(value: unknown): unknown { if (Array.isArray(value)) { - return value.map((item: any) => convertArrayItem(item)) + return value.map((item: unknown) => convertArrayItem(item)) } if (typeof value === 'string' && !isVariableReference(value)) { @@ -97,25 +97,34 @@ function convertArrayValue(value: any): any { return value } -function convertArrayItem(item: any): any { - if (typeof item !== 'object' || !item.type) { +function convertArrayItem(item: unknown): unknown { + if (typeof item !== 'object' || item === null || !('type' in item)) { return item } - if (item.type === 'object' && Array.isArray(item.value)) { - return convertBuilderDataToJson(item.value) + const record = item as Record + if (typeof record.type !== 'string') { + return item } - if (item.type === 'array' && Array.isArray(item.value)) { - return item.value.map((subItem: any) => - typeof subItem === 'object' && subItem.type ? subItem.value : subItem + const typed = record as { type: string; value: unknown } + + if (typed.type === 'object' && Array.isArray(typed.value)) { + return convertBuilderDataToJson(typed.value as JSONProperty[]) + } + + if (typed.type === 'array' && Array.isArray(typed.value)) { + return (typed.value as unknown[]).map((subItem: unknown) => + typeof subItem === 'object' && subItem !== null && 'value' in subItem + ? (subItem as { value: unknown }).value + : subItem ) } - return item.value + return typed.value } -function convertNumberValue(value: any): any { +function convertNumberValue(value: unknown): unknown { if (isVariableReference(value)) { return value } @@ -124,7 +133,7 @@ function convertNumberValue(value: any): any { return Number.isNaN(numValue) ? value : numValue } -function convertBooleanValue(value: any): any { +function convertBooleanValue(value: unknown): unknown { if (isVariableReference(value)) { return value } @@ -132,7 +141,7 @@ function convertBooleanValue(value: any): any { return value === 'true' || value === true } -function tryParseJson(jsonString: string, fallback: any): any { +function tryParseJson(jsonString: string, fallback: unknown): unknown { try { return JSON.parse(jsonString) } catch { @@ -140,7 +149,7 @@ function tryParseJson(jsonString: string, fallback: any): any { } } -function isVariableReference(value: any): boolean { +function isVariableReference(value: unknown): boolean { return ( typeof value === 'string' && value.trim().startsWith(REFERENCE.START) && diff --git a/apps/sim/executor/utils/iteration-context.test.ts b/apps/sim/executor/utils/iteration-context.test.ts new file mode 100644 index 0000000000..76583ca481 --- /dev/null +++ b/apps/sim/executor/utils/iteration-context.test.ts @@ -0,0 +1,585 @@ +/** + * @vitest-environment node + */ +import { describe, expect, it } from 'vitest' +import type { ExecutionContext } from '@/executor/types' +import { + buildContainerIterationContext, + buildUnifiedParentIterations, + getIterationContext, + type IterationNodeMetadata, +} from './iteration-context' + +function makeCtx(overrides: Partial = {}): ExecutionContext { + return { + workflowId: 'wf-1', + executionId: 'exec-1', + workspaceId: 'ws-1', + userId: 'user-1', + blockStates: {}, + blockLogs: [], + executedBlocks: [], + environmentVariables: {}, + decisions: { router: new Map(), condition: new Map() }, + completedLoops: new Set(), + activeExecutionPath: [], + executionOrder: 0, + ...overrides, + } as unknown as ExecutionContext +} + +describe('getIterationContext', () => { + it('returns undefined for undefined metadata', () => { + const ctx = makeCtx() + expect(getIterationContext(ctx, undefined)).toBeUndefined() + }) + + it('resolves parallel branch metadata', () => { + const ctx = makeCtx({ + parallelExecutions: new Map([ + [ + 'p1', + { + parallelId: 'p1', + totalBranches: 3, + branchOutputs: new Map(), + completedCount: 0, + totalExpectedNodes: 3, + }, + ], + ]), + }) + const metadata: IterationNodeMetadata = { + branchIndex: 1, + branchTotal: 3, + parallelId: 'p1', + } + const result = getIterationContext(ctx, metadata) + expect(result).toEqual({ + iterationCurrent: 1, + iterationTotal: 3, + iterationType: 'parallel', + iterationContainerId: 'p1', + }) + }) + + it('resolves loop node metadata', () => { + const ctx = makeCtx({ + loopExecutions: new Map([ + [ + 'l1', + { + iteration: 2, + maxIterations: 5, + currentIterationOutputs: new Map(), + allIterationOutputs: [], + }, + ], + ]), + }) + const metadata: IterationNodeMetadata = { + isLoopNode: true, + loopId: 'l1', + } + const result = getIterationContext(ctx, metadata) + expect(result).toEqual({ + iterationCurrent: 2, + iterationTotal: 5, + iterationType: 'loop', + iterationContainerId: 'l1', + }) + }) +}) + +describe('buildUnifiedParentIterations', () => { + it('returns empty array when no parent maps exist', () => { + const ctx = makeCtx() + expect(buildUnifiedParentIterations(ctx, 'some-id')).toEqual([]) + }) + + it('resolves loop-in-loop parent chain', () => { + const ctx = makeCtx({ + subflowParentMap: new Map([['inner-loop', { parentId: 'outer-loop', parentType: 'loop' }]]), + loopExecutions: new Map([ + [ + 'outer-loop', + { + iteration: 1, + maxIterations: 3, + currentIterationOutputs: new Map(), + allIterationOutputs: [], + }, + ], + ]), + }) + const result = buildUnifiedParentIterations(ctx, 'inner-loop') + expect(result).toEqual([ + { + iterationCurrent: 1, + iterationTotal: 3, + iterationType: 'loop', + iterationContainerId: 'outer-loop', + }, + ]) + }) + + it('resolves parallel-in-parallel parent chain', () => { + const ctx = makeCtx({ + subflowParentMap: new Map([ + ['inner-p__obranch-2', { parentId: 'outer-p', parentType: 'parallel' }], + ]), + parallelExecutions: new Map([ + [ + 'outer-p', + { + parallelId: 'outer-p', + totalBranches: 4, + branchOutputs: new Map(), + completedCount: 0, + totalExpectedNodes: 4, + }, + ], + ]), + }) + const result = buildUnifiedParentIterations(ctx, 'inner-p__obranch-2') + expect(result).toEqual([ + { + iterationCurrent: 2, + iterationTotal: 4, + iterationType: 'parallel', + iterationContainerId: 'outer-p', + }, + ]) + }) + + it('resolves loop-in-parallel (cross-type nesting)', () => { + const ctx = makeCtx({ + subflowParentMap: new Map([ + ['loop-1__obranch-1', { parentId: 'parallel-1', parentType: 'parallel' }], + ]), + parallelExecutions: new Map([ + [ + 'parallel-1', + { + parallelId: 'parallel-1', + totalBranches: 5, + branchOutputs: new Map(), + completedCount: 0, + totalExpectedNodes: 5, + }, + ], + ]), + }) + const result = buildUnifiedParentIterations(ctx, 'loop-1__obranch-1') + expect(result).toEqual([ + { + iterationCurrent: 1, + iterationTotal: 5, + iterationType: 'parallel', + iterationContainerId: 'parallel-1', + }, + ]) + }) + + it('resolves parallel-in-loop (cross-type nesting)', () => { + const ctx = makeCtx({ + subflowParentMap: new Map([['parallel-1', { parentId: 'loop-1', parentType: 'loop' }]]), + loopExecutions: new Map([ + [ + 'loop-1', + { + iteration: 3, + maxIterations: 5, + currentIterationOutputs: new Map(), + allIterationOutputs: [], + }, + ], + ]), + }) + const result = buildUnifiedParentIterations(ctx, 'parallel-1') + expect(result).toEqual([ + { + iterationCurrent: 3, + iterationTotal: 5, + iterationType: 'loop', + iterationContainerId: 'loop-1', + }, + ]) + }) + + it('resolves deep cross-type nesting: parallel → loop → parallel', () => { + const ctx = makeCtx({ + subflowParentMap: new Map([ + ['inner-p', { parentId: 'mid-loop', parentType: 'loop' }], + ['mid-loop', { parentId: 'outer-p', parentType: 'parallel' }], + ['mid-loop__obranch-2', { parentId: 'outer-p', parentType: 'parallel' }], + ]), + loopExecutions: new Map([ + [ + 'mid-loop', + { + iteration: 1, + maxIterations: 4, + currentIterationOutputs: new Map(), + allIterationOutputs: [], + }, + ], + ]), + parallelExecutions: new Map([ + [ + 'outer-p', + { + parallelId: 'outer-p', + totalBranches: 3, + branchOutputs: new Map(), + completedCount: 0, + totalExpectedNodes: 3, + }, + ], + ]), + }) + const result = buildUnifiedParentIterations(ctx, 'inner-p') + expect(result).toEqual([ + { + iterationCurrent: 0, + iterationTotal: 3, + iterationType: 'parallel', + iterationContainerId: 'outer-p', + }, + { + iterationCurrent: 1, + iterationTotal: 4, + iterationType: 'loop', + iterationContainerId: 'mid-loop', + }, + ]) + }) + + it('resolves 3-level parallel nesting with branchIndex entries', () => { + // P1 → P2 → P3, with P2__obranch-1 and P3__clone0__obranch-1 + const ctx = makeCtx({ + subflowParentMap: new Map([ + ['P2', { parentId: 'P1', parentType: 'parallel' }], + ['P3', { parentId: 'P2', parentType: 'parallel' }], + ['P2__obranch-1', { parentId: 'P1', parentType: 'parallel', branchIndex: 1 }], + [ + 'P3__clone0__obranch-1', + { parentId: 'P2__obranch-1', parentType: 'parallel', branchIndex: 0 }, + ], + ['P3__obranch-1', { parentId: 'P2', parentType: 'parallel', branchIndex: 1 }], + ]), + parallelExecutions: new Map([ + [ + 'P1', + { + parallelId: 'P1', + totalBranches: 2, + branchOutputs: new Map(), + completedCount: 0, + totalExpectedNodes: 2, + }, + ], + [ + 'P2', + { + parallelId: 'P2', + totalBranches: 2, + branchOutputs: new Map(), + completedCount: 0, + totalExpectedNodes: 2, + }, + ], + [ + 'P2__obranch-1', + { + parallelId: 'P2__obranch-1', + totalBranches: 2, + branchOutputs: new Map(), + completedCount: 0, + totalExpectedNodes: 2, + }, + ], + ]), + }) + + // P3 (original): inside P2 branch 0, inside P1 branch 0 + expect(buildUnifiedParentIterations(ctx, 'P3')).toEqual([ + { + iterationCurrent: 0, + iterationTotal: 2, + iterationType: 'parallel', + iterationContainerId: 'P1', + }, + { + iterationCurrent: 0, + iterationTotal: 2, + iterationType: 'parallel', + iterationContainerId: 'P2', + }, + ]) + + // P3__obranch-1 (runtime clone): inside P2 branch 1, inside P1 branch 0 + expect(buildUnifiedParentIterations(ctx, 'P3__obranch-1')).toEqual([ + { + iterationCurrent: 0, + iterationTotal: 2, + iterationType: 'parallel', + iterationContainerId: 'P1', + }, + { + iterationCurrent: 1, + iterationTotal: 2, + iterationType: 'parallel', + iterationContainerId: 'P2', + }, + ]) + + // P3__clone0__obranch-1 (pre-expansion clone): inside P2__obranch-1 branch 0, inside P1 branch 1 + expect(buildUnifiedParentIterations(ctx, 'P3__clone0__obranch-1')).toEqual([ + { + iterationCurrent: 1, + iterationTotal: 2, + iterationType: 'parallel', + iterationContainerId: 'P1', + }, + { + iterationCurrent: 0, + iterationTotal: 2, + iterationType: 'parallel', + iterationContainerId: 'P2__obranch-1', + }, + ]) + }) + + it('includes parent iterations in getIterationContext for loop-in-parallel', () => { + const ctx = makeCtx({ + subflowParentMap: new Map([ + ['loop-1__obranch-2', { parentId: 'parallel-1', parentType: 'parallel' }], + ]), + parallelExecutions: new Map([ + [ + 'parallel-1', + { + parallelId: 'parallel-1', + totalBranches: 5, + branchOutputs: new Map(), + completedCount: 0, + totalExpectedNodes: 5, + }, + ], + ]), + loopExecutions: new Map([ + [ + 'loop-1__obranch-2', + { + iteration: 3, + maxIterations: 5, + currentIterationOutputs: new Map(), + allIterationOutputs: [], + }, + ], + ]), + }) + + const metadata: IterationNodeMetadata = { + isLoopNode: true, + loopId: 'loop-1__obranch-2', + } + const result = getIterationContext(ctx, metadata) + expect(result).toEqual({ + iterationCurrent: 3, + iterationTotal: 5, + iterationType: 'loop', + iterationContainerId: 'loop-1__obranch-2', + parentIterations: [ + { + iterationCurrent: 2, + iterationTotal: 5, + iterationType: 'parallel', + iterationContainerId: 'parallel-1', + }, + ], + }) + }) + + it('includes parent iterations in getIterationContext for parallel-in-loop', () => { + const ctx = makeCtx({ + subflowParentMap: new Map([['parallel-1', { parentId: 'loop-1', parentType: 'loop' }]]), + loopExecutions: new Map([ + [ + 'loop-1', + { + iteration: 2, + maxIterations: 5, + currentIterationOutputs: new Map(), + allIterationOutputs: [], + }, + ], + ]), + parallelExecutions: new Map([ + [ + 'parallel-1', + { + parallelId: 'parallel-1', + totalBranches: 3, + branchOutputs: new Map(), + completedCount: 0, + totalExpectedNodes: 3, + }, + ], + ]), + }) + + const metadata: IterationNodeMetadata = { + branchIndex: 1, + branchTotal: 3, + parallelId: 'parallel-1', + } + const result = getIterationContext(ctx, metadata) + expect(result).toEqual({ + iterationCurrent: 1, + iterationTotal: 3, + iterationType: 'parallel', + iterationContainerId: 'parallel-1', + parentIterations: [ + { + iterationCurrent: 2, + iterationTotal: 5, + iterationType: 'loop', + iterationContainerId: 'loop-1', + }, + ], + }) + }) +}) + +describe('buildContainerIterationContext', () => { + it('returns undefined when no parent map exists', () => { + const ctx = makeCtx() + expect(buildContainerIterationContext(ctx, 'loop-1')).toBeUndefined() + }) + + it('returns undefined when container is not in parent map', () => { + const ctx = makeCtx({ + subflowParentMap: new Map(), + }) + expect(buildContainerIterationContext(ctx, 'loop-1')).toBeUndefined() + }) + + it('resolves loop nested inside parallel', () => { + const ctx = makeCtx({ + subflowParentMap: new Map([ + ['loop-1__obranch-2', { parentId: 'parallel-1', parentType: 'parallel' }], + ]), + parallelExecutions: new Map([ + [ + 'parallel-1', + { + parallelId: 'parallel-1', + totalBranches: 5, + branchOutputs: new Map(), + completedCount: 0, + totalExpectedNodes: 5, + }, + ], + ]), + }) + const result = buildContainerIterationContext(ctx, 'loop-1__obranch-2') + expect(result).toEqual({ + iterationCurrent: 2, + iterationTotal: 5, + iterationType: 'parallel', + iterationContainerId: 'parallel-1', + }) + }) + + it('resolves parallel nested inside loop', () => { + const ctx = makeCtx({ + subflowParentMap: new Map([['parallel-1', { parentId: 'loop-1', parentType: 'loop' }]]), + loopExecutions: new Map([ + [ + 'loop-1', + { + iteration: 3, + maxIterations: 10, + currentIterationOutputs: new Map(), + allIterationOutputs: [], + }, + ], + ]), + }) + const result = buildContainerIterationContext(ctx, 'parallel-1') + expect(result).toEqual({ + iterationCurrent: 3, + iterationTotal: 10, + iterationType: 'loop', + iterationContainerId: 'loop-1', + }) + }) + + it('returns undefined when parent scope is missing', () => { + const ctx = makeCtx({ + subflowParentMap: new Map([['loop-1', { parentId: 'parallel-1', parentType: 'parallel' }]]), + parallelExecutions: new Map(), + }) + expect(buildContainerIterationContext(ctx, 'loop-1')).toBeUndefined() + }) + + it('resolves pre-expansion clone with explicit branchIndex', () => { + // P1 → P2 → P3: P3__clone0__obranch-1 is pre-cloned inside P2__obranch-1 + const ctx = makeCtx({ + subflowParentMap: new Map([ + [ + 'P3__clone0__obranch-1', + { parentId: 'P2__obranch-1', parentType: 'parallel', branchIndex: 0 }, + ], + ]), + parallelExecutions: new Map([ + [ + 'P2__obranch-1', + { + parallelId: 'P2__obranch-1', + totalBranches: 5, + branchOutputs: new Map(), + completedCount: 0, + totalExpectedNodes: 5, + }, + ], + ]), + }) + const result = buildContainerIterationContext(ctx, 'P3__clone0__obranch-1') + expect(result).toEqual({ + iterationCurrent: 0, + iterationTotal: 5, + iterationType: 'parallel', + iterationContainerId: 'P2__obranch-1', + }) + }) + + it('uses branch index 0 for non-cloned container in parallel parent', () => { + const ctx = makeCtx({ + subflowParentMap: new Map([ + ['inner-loop', { parentId: 'outer-parallel', parentType: 'parallel' }], + ]), + parallelExecutions: new Map([ + [ + 'outer-parallel', + { + parallelId: 'outer-parallel', + totalBranches: 3, + branchOutputs: new Map(), + completedCount: 0, + totalExpectedNodes: 3, + }, + ], + ]), + }) + const result = buildContainerIterationContext(ctx, 'inner-loop') + expect(result).toEqual({ + iterationCurrent: 0, + iterationTotal: 3, + iterationType: 'parallel', + iterationContainerId: 'outer-parallel', + }) + }) +}) diff --git a/apps/sim/executor/utils/iteration-context.ts b/apps/sim/executor/utils/iteration-context.ts new file mode 100644 index 0000000000..edf9a0e517 --- /dev/null +++ b/apps/sim/executor/utils/iteration-context.ts @@ -0,0 +1,179 @@ +import { DEFAULTS } from '@/executor/constants' +import type { NodeMetadata } from '@/executor/dag/types' +import type { IterationContext, ParentIteration } from '@/executor/execution/types' +import type { ExecutionContext } from '@/executor/types' +import { extractOuterBranchIndex, findEffectiveContainerId } from '@/executor/utils/subflow-utils' + +/** Maximum ancestor depth to prevent runaway traversal in deeply nested subflows. */ +const MAX_PARENT_DEPTH = DEFAULTS.MAX_NESTING_DEPTH + +/** + * Subset of {@link NodeMetadata} needed for iteration context resolution. + * Compatible with both DAGNode.metadata and inline metadata objects. + */ +export type IterationNodeMetadata = Pick< + NodeMetadata, + 'loopId' | 'parallelId' | 'branchIndex' | 'branchTotal' | 'isLoopNode' +> + +/** + * Resolves the iteration context for a node based on its metadata and execution state. + * Handles both parallel (branch) and loop iteration contexts, including cross-type + * nesting (loop-in-parallel, parallel-in-loop) via the unified subflow parent map. + */ +export function getIterationContext( + ctx: ExecutionContext, + metadata: IterationNodeMetadata | undefined +): IterationContext | undefined { + if (!metadata) return undefined + + if (metadata.branchIndex !== undefined && metadata.branchTotal !== undefined) { + const parentIterations = metadata.parallelId + ? buildUnifiedParentIterations(ctx, metadata.parallelId) + : [] + return { + iterationCurrent: metadata.branchIndex, + iterationTotal: metadata.branchTotal, + iterationType: 'parallel', + iterationContainerId: metadata.parallelId, + ...(parentIterations.length > 0 && { parentIterations }), + } + } + + if (metadata.isLoopNode && metadata.loopId) { + const loopScope = ctx.loopExecutions?.get(metadata.loopId) + if (loopScope && loopScope.iteration !== undefined) { + const parentIterations = buildUnifiedParentIterations(ctx, metadata.loopId) + return { + iterationCurrent: loopScope.iteration, + iterationTotal: loopScope.maxIterations, + iterationType: 'loop', + iterationContainerId: metadata.loopId, + ...(parentIterations.length > 0 && { parentIterations }), + } + } + } + + return undefined +} + +/** + * Builds a single-level iteration context for a container (loop/parallel) that is + * nested inside a parent subflow. Used by orchestrators when emitting onBlockComplete + * for container sentinel nodes. + */ +export function buildContainerIterationContext( + ctx: ExecutionContext, + containerId: string +): IterationContext | undefined { + const parentEntry = ctx.subflowParentMap?.get(containerId) + if (!parentEntry) return undefined + + if (parentEntry.parentType === 'parallel') { + // Use stored parentId directly when branchIndex is available (set during expansion), + // otherwise fall back to findEffectiveContainerId for backward compatibility. + const hasBranchIndex = parentEntry.branchIndex !== undefined + const effectiveParentId = hasBranchIndex + ? parentEntry.parentId + : ctx.parallelExecutions + ? findEffectiveContainerId(parentEntry.parentId, containerId, ctx.parallelExecutions) + : parentEntry.parentId + const parentScope = ctx.parallelExecutions?.get(effectiveParentId) + if (parentScope) { + return { + iterationCurrent: hasBranchIndex + ? parentEntry.branchIndex! + : (extractOuterBranchIndex(containerId) ?? 0), + iterationTotal: parentScope.totalBranches, + iterationType: 'parallel', + iterationContainerId: effectiveParentId, + } + } + } else if (parentEntry.parentType === 'loop') { + const effectiveParentId = ctx.loopExecutions + ? findEffectiveContainerId(parentEntry.parentId, containerId, ctx.loopExecutions) + : parentEntry.parentId + const parentScope = ctx.loopExecutions?.get(effectiveParentId) + if (parentScope && parentScope.iteration !== undefined) { + return { + iterationCurrent: parentScope.iteration, + iterationTotal: parentScope.maxIterations, + iterationType: 'loop', + iterationContainerId: effectiveParentId, + } + } + } + return undefined +} + +/** + * Walks the unified subflow parent map to build the full ancestor iteration chain, + * handling all nesting combinations (loop-in-loop, parallel-in-parallel, + * loop-in-parallel, parallel-in-loop). + * + * Returns an array of parent iteration contexts, ordered from outermost to innermost. + */ +export function buildUnifiedParentIterations( + ctx: ExecutionContext, + subflowId: string +): ParentIteration[] { + if (!ctx.subflowParentMap) { + return [] + } + + const parents: ParentIteration[] = [] + const visited = new Set() + let currentId = subflowId + + while ( + ctx.subflowParentMap.has(currentId) && + !visited.has(currentId) && + visited.size < MAX_PARENT_DEPTH + ) { + visited.add(currentId) + const entry = ctx.subflowParentMap.get(currentId)! + const { parentId, parentType } = entry + + if (parentType === 'loop') { + // Resolve the effective (possibly cloned) loop ID — at runtime the scope + // may live under a cloned ID like `mid-loop__obranch-2` rather than `mid-loop` + const effectiveParentId = ctx.loopExecutions + ? findEffectiveContainerId(parentId, currentId, ctx.loopExecutions) + : parentId + const parentScope = ctx.loopExecutions?.get(effectiveParentId) + if (parentScope && parentScope.iteration !== undefined) { + parents.unshift({ + iterationCurrent: parentScope.iteration, + iterationTotal: parentScope.maxIterations, + iterationType: 'loop', + iterationContainerId: effectiveParentId, + }) + } + } else { + // Use stored parentId directly when branchIndex is available (set during expansion), + // otherwise fall back to findEffectiveContainerId for backward compatibility. + const hasBranchIndex = entry.branchIndex !== undefined + const effectiveParentId = hasBranchIndex + ? parentId + : ctx.parallelExecutions + ? findEffectiveContainerId(parentId, currentId, ctx.parallelExecutions) + : parentId + const parentScope = ctx.parallelExecutions?.get(effectiveParentId) + if (parentScope) { + const outerBranchIndex = hasBranchIndex + ? entry.branchIndex! + : (extractOuterBranchIndex(currentId) ?? 0) + parents.unshift({ + iterationCurrent: outerBranchIndex, + iterationTotal: parentScope.totalBranches, + iterationType: 'parallel', + iterationContainerId: effectiveParentId, + }) + } + } + + currentId = parentId + } + + return parents +} diff --git a/apps/sim/executor/utils/parallel-expansion.test.ts b/apps/sim/executor/utils/parallel-expansion.test.ts new file mode 100644 index 0000000000..16daa93e4e --- /dev/null +++ b/apps/sim/executor/utils/parallel-expansion.test.ts @@ -0,0 +1,316 @@ +/** + * @vitest-environment node + */ +import { loggerMock } from '@sim/testing' +import { describe, expect, it, vi } from 'vitest' +import { BlockType } from '@/executor/constants' +import { DAGBuilder } from '@/executor/dag/builder' +import { EdgeManager } from '@/executor/execution/edge-manager' +import { ParallelExpander } from '@/executor/utils/parallel-expansion' +import { + buildBranchNodeId, + buildParallelSentinelEndId, + buildParallelSentinelStartId, + stripCloneSuffixes, +} from '@/executor/utils/subflow-utils' +import type { SerializedBlock, SerializedWorkflow } from '@/serializer/types' + +vi.mock('@sim/logger', () => loggerMock) + +function createBlock(id: string, metadataId: string): SerializedBlock { + return { + id, + position: { x: 0, y: 0 }, + config: { tool: 'noop', params: {} }, + inputs: {}, + outputs: {}, + metadata: { id: metadataId, name: id }, + enabled: true, + } +} + +describe('Nested parallel expansion + edge resolution', () => { + it('outer parallel expansion clones inner subflow per branch and edge manager resolves correctly', () => { + const outerParallelId = 'outer-parallel' + const innerParallelId = 'inner-parallel' + const functionId = 'func-1' + + const workflow: SerializedWorkflow = { + version: '1', + blocks: [ + createBlock('start', BlockType.STARTER), + createBlock(outerParallelId, BlockType.PARALLEL), + createBlock(innerParallelId, BlockType.PARALLEL), + createBlock(functionId, BlockType.FUNCTION), + ], + connections: [ + { source: 'start', target: outerParallelId }, + { + source: outerParallelId, + target: innerParallelId, + sourceHandle: 'parallel-start-source', + }, + { + source: innerParallelId, + target: functionId, + sourceHandle: 'parallel-start-source', + }, + ], + loops: {}, + parallels: { + [innerParallelId]: { + id: innerParallelId, + nodes: [functionId], + count: 3, + parallelType: 'count', + }, + [outerParallelId]: { + id: outerParallelId, + nodes: [innerParallelId], + count: 2, + parallelType: 'count', + }, + }, + } + + // Step 1: Build the DAG + const builder = new DAGBuilder() + const dag = builder.build(workflow) + + const outerStartId = buildParallelSentinelStartId(outerParallelId) + const outerEndId = buildParallelSentinelEndId(outerParallelId) + const innerStartId = buildParallelSentinelStartId(innerParallelId) + const innerEndId = buildParallelSentinelEndId(innerParallelId) + + // Verify DAG construction: start → outer-sentinel-start + const startNode = dag.nodes.get('start')! + const startTargets = Array.from(startNode.outgoingEdges.values()).map((e) => e.target) + expect(startTargets).toContain(outerStartId) + + // Step 2: Simulate runtime expansion of outer parallel (count=2) + const expander = new ParallelExpander() + const outerResult = expander.expandParallel(dag, outerParallelId, 2) + + // After expansion, outer-sentinel-start should point to 2 entry nodes: + // branch 0 uses original inner-sentinel-start, branch 1 uses cloned sentinel + const outerStart = dag.nodes.get(outerStartId)! + const outerStartTargets = Array.from(outerStart.outgoingEdges.values()).map((e) => e.target) + expect(outerStartTargets).toHaveLength(2) + expect(outerStartTargets).toContain(innerStartId) // branch 0 + + // Verify cloned subflow info + expect(outerResult.clonedSubflows).toHaveLength(1) + expect(outerResult.clonedSubflows[0].originalId).toBe(innerParallelId) + expect(outerResult.clonedSubflows[0].outerBranchIndex).toBe(1) + + const clonedInnerParallelId = outerResult.clonedSubflows[0].clonedId + const clonedInnerStartId = buildParallelSentinelStartId(clonedInnerParallelId) + const clonedInnerEndId = buildParallelSentinelEndId(clonedInnerParallelId) + + expect(outerStartTargets).toContain(clonedInnerStartId) // branch 1 + + // Verify cloned parallel config was registered + expect(dag.parallelConfigs.has(clonedInnerParallelId)).toBe(true) + const clonedConfig = dag.parallelConfigs.get(clonedInnerParallelId)! + expect(clonedConfig.count).toBe(3) + expect(clonedConfig.nodes).toHaveLength(1) + + // inner-sentinel-end → outer-sentinel-end (branch 0) + const innerEnd = dag.nodes.get(innerEndId)! + const innerEndTargets = Array.from(innerEnd.outgoingEdges.values()).map((e) => e.target) + expect(innerEndTargets).toContain(outerEndId) + + // cloned inner sentinel-end → outer-sentinel-end (branch 1) + const clonedInnerEnd = dag.nodes.get(clonedInnerEndId)! + const clonedInnerEndTargets = Array.from(clonedInnerEnd.outgoingEdges.values()).map( + (e) => e.target + ) + expect(clonedInnerEndTargets).toContain(outerEndId) + + // Entry/terminal nodes from expansion + expect(outerResult.entryNodes).toContain(innerStartId) + expect(outerResult.entryNodes).toContain(clonedInnerStartId) + expect(outerResult.terminalNodes).toContain(innerEndId) + expect(outerResult.terminalNodes).toContain(clonedInnerEndId) + + // Step 3: Verify edge manager resolves ready nodes after outer-sentinel-start completes + const edgeManager = new EdgeManager(dag) + const readyAfterOuterStart = edgeManager.processOutgoingEdges( + outerStart, + { sentinelStart: true }, + false + ) + expect(readyAfterOuterStart).toContain(innerStartId) + expect(readyAfterOuterStart).toContain(clonedInnerStartId) + + // Step 4: Expand inner parallel (branch 0's inner) with count=3 + expander.expandParallel(dag, innerParallelId, 3) + + // Inner sentinel-start should now point to 3 branch nodes + const innerStart = dag.nodes.get(innerStartId)! + const innerStartTargets = Array.from(innerStart.outgoingEdges.values()).map((e) => e.target) + expect(innerStartTargets).toHaveLength(3) + + const branch0 = buildBranchNodeId(functionId, 0) + const branch1 = buildBranchNodeId(functionId, 1) + const branch2 = buildBranchNodeId(functionId, 2) + expect(innerStartTargets).toContain(branch0) + expect(innerStartTargets).toContain(branch1) + expect(innerStartTargets).toContain(branch2) + + // Step 5: Verify edge manager resolves branch nodes after inner-sentinel-start + const readyAfterInnerStart = edgeManager.processOutgoingEdges( + innerStart, + { sentinelStart: true }, + false + ) + expect(readyAfterInnerStart).toContain(branch0) + expect(readyAfterInnerStart).toContain(branch1) + expect(readyAfterInnerStart).toContain(branch2) + + // Step 6: Simulate branch completions → inner-sentinel-end becomes ready + const branch0Node = dag.nodes.get(branch0)! + const branch1Node = dag.nodes.get(branch1)! + const branch2Node = dag.nodes.get(branch2)! + + edgeManager.processOutgoingEdges(branch0Node, {}, false) + edgeManager.processOutgoingEdges(branch1Node, {}, false) + const readyAfterBranch2 = edgeManager.processOutgoingEdges(branch2Node, {}, false) + expect(readyAfterBranch2).toContain(innerEndId) + + // Step 7: inner-sentinel-end completes → outer-sentinel-end becomes ready + // (only if both branches are done — cloned branch must also complete) + const readyAfterInnerEnd = edgeManager.processOutgoingEdges( + innerEnd, + { sentinelEnd: true, selectedRoute: 'parallel_exit' }, + false + ) + // outer-sentinel-end has 2 incoming (innerEnd + clonedInnerEnd), not ready yet + expect(readyAfterInnerEnd).not.toContain(outerEndId) + + // Expand and complete cloned inner parallel (branch 1's inner) + const clonedBlockId = clonedConfig.nodes![0] + expander.expandParallel(dag, clonedInnerParallelId, 3) + const clonedInnerStart = dag.nodes.get(clonedInnerStartId)! + const clonedBranch0 = buildBranchNodeId(clonedBlockId, 0) + const clonedBranch1 = buildBranchNodeId(clonedBlockId, 1) + const clonedBranch2 = buildBranchNodeId(clonedBlockId, 2) + + edgeManager.processOutgoingEdges(clonedInnerStart, { sentinelStart: true }, false) + edgeManager.processOutgoingEdges(dag.nodes.get(clonedBranch0)!, {}, false) + edgeManager.processOutgoingEdges(dag.nodes.get(clonedBranch1)!, {}, false) + edgeManager.processOutgoingEdges(dag.nodes.get(clonedBranch2)!, {}, false) + + const readyAfterClonedInnerEnd = edgeManager.processOutgoingEdges( + clonedInnerEnd, + { sentinelEnd: true, selectedRoute: 'parallel_exit' }, + false + ) + // Now both branches done → outer-sentinel-end becomes ready + expect(readyAfterClonedInnerEnd).toContain(outerEndId) + }) + + it('3-level nesting: pre-expansion clone IDs do not collide with runtime expansion', () => { + const p1 = 'p1' + const p2 = 'p2' + const p3 = 'p3' + const leafBlock = 'leaf' + + const workflow: SerializedWorkflow = { + version: '1', + blocks: [ + createBlock('start', BlockType.STARTER), + createBlock(p1, BlockType.PARALLEL), + createBlock(p2, BlockType.PARALLEL), + createBlock(p3, BlockType.PARALLEL), + createBlock(leafBlock, BlockType.FUNCTION), + ], + connections: [ + { source: 'start', target: p1 }, + { source: p1, target: p2, sourceHandle: 'parallel-start-source' }, + { source: p2, target: p3, sourceHandle: 'parallel-start-source' }, + { source: p3, target: leafBlock, sourceHandle: 'parallel-start-source' }, + ], + loops: {}, + parallels: { + [p3]: { id: p3, nodes: [leafBlock], count: 2, parallelType: 'count' }, + [p2]: { id: p2, nodes: [p3], count: 2, parallelType: 'count' }, + [p1]: { id: p1, nodes: [p2], count: 2, parallelType: 'count' }, + }, + } + + const builder = new DAGBuilder() + const dag = builder.build(workflow) + const expander = new ParallelExpander() + + // Step 1: Expand P1 (outermost) — this pre-clones P2 and recursively P3 + const p1Result = expander.expandParallel(dag, p1, 2) + + // P1 should have cloned P2 (and recursively P3 inside it) + const p2Clone = p1Result.clonedSubflows.find((c) => c.originalId === p2)! + expect(p2Clone).toBeDefined() + expect(p2Clone.clonedId).toBe('p2__obranch-1') + + // P3 should also be cloned (inside P2__obranch-1) with a __clone prefix + const p3Clone = p1Result.clonedSubflows.find((c) => c.originalId === p3)! + expect(p3Clone).toBeDefined() + expect(p3Clone.clonedId).toMatch(/^p3__clone\d+__obranch-1$/) + expect(stripCloneSuffixes(p3Clone.clonedId)).toBe('p3') + + // Step 2: Expand P2 (original, branch 0 of P1) — this creates P3__obranch-1 at runtime + const p2Result = expander.expandParallel(dag, p2, 2) + + // P2 should clone P3 as P3__obranch-1 (standard runtime naming) + const p3RuntimeClone = p2Result.clonedSubflows.find((c) => c.originalId === p3)! + expect(p3RuntimeClone).toBeDefined() + expect(p3RuntimeClone.clonedId).toBe('p3__obranch-1') + + // Key assertion: P3__obranch-1 (runtime) !== P3__clone*__obranch-1 (pre-expansion) + expect(p3RuntimeClone.clonedId).not.toBe(p3Clone.clonedId) + + // Both P3 configs should exist independently in the DAG + expect(dag.parallelConfigs.has(p3RuntimeClone.clonedId)).toBe(true) + expect(dag.parallelConfigs.has(p3Clone.clonedId)).toBe(true) + + // Step 3: Expand P2__obranch-1 (cloned, branch 1 of P1) + // Its inner P3 is the pre-cloned variant P3__clone*__obranch-1 + const p2ClonedConfig = dag.parallelConfigs.get(p2Clone.clonedId)! + const p3InsideP2Clone = p2ClonedConfig.nodes![0] + expect(p3InsideP2Clone).toBe(p3Clone.clonedId) + + const p2CloneResult = expander.expandParallel(dag, p2Clone.clonedId, 2) + + // P2__obranch-1 should clone its P3 (the pre-cloned variant) with __obranch-1 suffix + const p3DeepClone = p2CloneResult.clonedSubflows.find((c) => c.originalId === p3Clone.clonedId)! + expect(p3DeepClone).toBeDefined() + // This ID should be unique (no collision with any earlier P3 clone) + expect(dag.parallelConfigs.has(p3DeepClone.clonedId)).toBe(true) + + // Step 4: Expand all P3 variants and verify no node collisions + const allP3Variants = [p3, p3RuntimeClone.clonedId, p3Clone.clonedId, p3DeepClone.clonedId] + const allLeafNodes = new Set() + + for (const p3Id of allP3Variants) { + const p3Config = dag.parallelConfigs.get(p3Id)! + const leafId = p3Config.nodes![0] + + const p3Result = expander.expandParallel(dag, p3Id, 2) + + // Each expansion creates branch nodes — verify they're unique + const branch0 = buildBranchNodeId(leafId, 0) + const branch1 = buildBranchNodeId(leafId, 1) + + expect(dag.nodes.has(branch0)).toBe(true) + expect(dag.nodes.has(branch1)).toBe(true) + + // No duplicate node IDs across all expansions + expect(allLeafNodes.has(branch0)).toBe(false) + expect(allLeafNodes.has(branch1)).toBe(false) + allLeafNodes.add(branch0) + allLeafNodes.add(branch1) + } + + // 4 P3 variants × 2 branches each = 8 unique leaf nodes + expect(allLeafNodes.size).toBe(8) + }) +}) diff --git a/apps/sim/executor/utils/parallel-expansion.ts b/apps/sim/executor/utils/parallel-expansion.ts index 00c458353b..6d59af91c8 100644 --- a/apps/sim/executor/utils/parallel-expansion.ts +++ b/apps/sim/executor/utils/parallel-expansion.ts @@ -1,22 +1,37 @@ import { createLogger } from '@sim/logger' +import { EDGE } from '@/executor/constants' import type { DAG, DAGNode } from '@/executor/dag/builder' import type { SerializedBlock } from '@/serializer/types' import { buildBranchNodeId, + buildClonedSubflowId, buildParallelSentinelEndId, buildParallelSentinelStartId, + buildSentinelEndId, + buildSentinelStartId, extractBaseBlockId, + isLoopSentinelNodeId, } from './subflow-utils' const logger = createLogger('ParallelExpansion') +export interface ClonedSubflowInfo { + clonedId: string + originalId: string + outerBranchIndex: number +} + export interface ExpansionResult { entryNodes: string[] terminalNodes: string[] allBranchNodes: string[] + clonedSubflows: ClonedSubflowInfo[] } export class ParallelExpander { + /** Monotonically increasing counter for generating unique pre-expansion clone IDs. */ + private cloneSeq = 0 + expandParallel( dag: DAG, parallelId: string, @@ -30,13 +45,27 @@ export class ParallelExpander { const blocksInParallel = config.nodes || [] if (blocksInParallel.length === 0) { - return { entryNodes: [], terminalNodes: [], allBranchNodes: [] } + return { entryNodes: [], terminalNodes: [], allBranchNodes: [], clonedSubflows: [] } } - const blocksSet = new Set(blocksInParallel) - const allBranchNodes: string[] = [] + // Separate nested subflow containers from regular expandable blocks. + // Nested parallels/loops have sentinel nodes instead of branch template nodes, + // so they cannot be cloned per-branch like regular blocks. + const regularBlocks: string[] = [] + const nestedSubflows: string[] = [] for (const blockId of blocksInParallel) { + if (dag.parallelConfigs.has(blockId) || dag.loopConfigs.has(blockId)) { + nestedSubflows.push(blockId) + } else { + regularBlocks.push(blockId) + } + } + + const regularSet = new Set(regularBlocks) + const allBranchNodes: string[] = [] + + for (const blockId of regularBlocks) { const templateId = buildBranchNodeId(blockId, 0) const templateNode = dag.nodes.get(templateId) @@ -65,14 +94,43 @@ export class ParallelExpander { } } - this.wireInternalEdges(dag, blocksInParallel, blocksSet, branchCount) + this.wireInternalEdges(dag, regularBlocks, regularSet, branchCount) - const { entryNodes, terminalNodes } = this.identifyBoundaryNodes( - dag, - blocksInParallel, - blocksSet, - branchCount - ) + const { entryNodes, terminalNodes } = + regularBlocks.length > 0 + ? this.identifyBoundaryNodes(dag, regularBlocks, regularSet, branchCount) + : { entryNodes: [] as string[], terminalNodes: [] as string[] } + + // Clone nested subflow graphs per outer branch so each branch runs independently. + // Branch 0 uses the original sentinel/template nodes; branches 1..N get full clones. + const clonedSubflows: ClonedSubflowInfo[] = [] + + for (const subflowId of nestedSubflows) { + const isParallel = dag.parallelConfigs.has(subflowId) + const startId = isParallel + ? buildParallelSentinelStartId(subflowId) + : buildSentinelStartId(subflowId) + const endId = isParallel + ? buildParallelSentinelEndId(subflowId) + : buildSentinelEndId(subflowId) + + // Branch 0 uses original nodes + if (dag.nodes.has(startId)) entryNodes.push(startId) + if (dag.nodes.has(endId)) terminalNodes.push(endId) + + // Branches 1..N clone the entire subflow graph (recursively for deep nesting) + for (let i = 1; i < branchCount; i++) { + const cloned = this.cloneNestedSubflow(dag, subflowId, i, clonedSubflows) + + entryNodes.push(cloned.startId) + terminalNodes.push(cloned.endId) + clonedSubflows.push({ + clonedId: cloned.clonedId, + originalId: subflowId, + outerBranchIndex: i, + }) + } + } this.wireSentinelEdges(dag, parallelId, entryNodes, terminalNodes, branchCount) @@ -80,10 +138,11 @@ export class ParallelExpander { parallelId, branchCount, blocksCount: blocksInParallel.length, + nestedSubflows: nestedSubflows.length, totalNodes: allBranchNodes.length, }) - return { entryNodes, terminalNodes, allBranchNodes } + return { entryNodes, terminalNodes, allBranchNodes, clonedSubflows } } private updateBranchMetadata( @@ -216,6 +275,207 @@ export class ParallelExpander { return false } + /** + * Generates a unique clone ID for pre-expansion cloning. + * + * Pre-expansion clones use `{originalId}__clone{N}__obranch-{branchIndex}` instead + * of the plain `{originalId}__obranch-{branchIndex}` used by runtime expansion. + * The `__clone{N}` segment (from a monotonic counter) prevents naming collisions + * when the original (branch-0) subflow later expands at runtime and creates + * `{child}__obranch-{branchIndex}`. + */ + private buildPreCloneId(originalId: string, outerBranchIndex: number): string { + return `${originalId}__clone${this.cloneSeq++}__obranch-${outerBranchIndex}` + } + + /** + * Clones an entire nested subflow graph for a specific outer branch. + * + * The top-level subflow gets a standard `__obranch-{N}` clone ID (needed by + * `findEffectiveContainerId` at runtime). All deeper children — both containers + * and regular blocks — receive unique `__clone{N}__obranch-{M}` IDs via + * {@link buildPreCloneId} to avoid collisions with runtime expansion. + */ + private cloneNestedSubflow( + dag: DAG, + subflowId: string, + outerBranchIndex: number, + clonedSubflows: ClonedSubflowInfo[] + ): { startId: string; endId: string; clonedId: string; idMap: Map } { + const clonedId = buildClonedSubflowId(subflowId, outerBranchIndex) + const { startId, endId, idMap } = this.cloneSubflowGraph( + dag, + subflowId, + clonedId, + outerBranchIndex, + clonedSubflows + ) + return { startId, endId, clonedId, idMap } + } + + /** + * Core recursive cloning: duplicates a subflow's sentinels, config, child blocks, + * and DAG nodes under the given `clonedId`. Nested containers are recursively + * cloned with unique pre-clone IDs. + */ + private cloneSubflowGraph( + dag: DAG, + originalId: string, + clonedId: string, + outerBranchIndex: number, + clonedSubflows: ClonedSubflowInfo[] + ): { startId: string; endId: string; idMap: Map } { + const isParallel = dag.parallelConfigs.has(originalId) + const config = isParallel + ? dag.parallelConfigs.get(originalId)! + : dag.loopConfigs.get(originalId)! + const blockIds = config.nodes || [] + const idMap = new Map() + + // Map sentinel nodes + const origStartId = isParallel + ? buildParallelSentinelStartId(originalId) + : buildSentinelStartId(originalId) + const origEndId = isParallel + ? buildParallelSentinelEndId(originalId) + : buildSentinelEndId(originalId) + const clonedStartId = isParallel + ? buildParallelSentinelStartId(clonedId) + : buildSentinelStartId(clonedId) + const clonedEndId = isParallel + ? buildParallelSentinelEndId(clonedId) + : buildSentinelEndId(clonedId) + + idMap.set(origStartId, clonedStartId) + idMap.set(origEndId, clonedEndId) + + // Process child blocks — recurse into nested containers, remap regular blocks + const clonedBlockIds: string[] = [] + + for (const blockId of blockIds) { + const isNestedParallel = dag.parallelConfigs.has(blockId) + const isNestedLoop = dag.loopConfigs.has(blockId) + + if (isNestedParallel || isNestedLoop) { + const nestedClonedId = this.buildPreCloneId(blockId, outerBranchIndex) + clonedBlockIds.push(nestedClonedId) + + const innerResult = this.cloneSubflowGraph( + dag, + blockId, + nestedClonedId, + outerBranchIndex, + clonedSubflows + ) + for (const [k, v] of innerResult.idMap) { + idMap.set(k, v) + } + + clonedSubflows.push({ + clonedId: nestedClonedId, + originalId: blockId, + outerBranchIndex, + }) + } else { + const clonedBlockId = this.buildPreCloneId(blockId, outerBranchIndex) + clonedBlockIds.push(clonedBlockId) + + if (isParallel) { + idMap.set(buildBranchNodeId(blockId, 0), buildBranchNodeId(clonedBlockId, 0)) + } else { + idMap.set(blockId, clonedBlockId) + } + } + } + + // Register cloned config + if (isParallel) { + dag.parallelConfigs.set(clonedId, { + ...dag.parallelConfigs.get(originalId)!, + id: clonedId, + nodes: clonedBlockIds, + }) + } else { + dag.loopConfigs.set(clonedId, { + ...dag.loopConfigs.get(originalId)!, + id: clonedId, + nodes: clonedBlockIds, + }) + } + + // Clone DAG nodes (sentinels + regular blocks) with remapped edges + const origNodeIds = [origStartId, origEndId] + for (const blockId of blockIds) { + if (dag.parallelConfigs.has(blockId) || dag.loopConfigs.has(blockId)) continue + if (isParallel) { + origNodeIds.push(buildBranchNodeId(blockId, 0)) + } else { + origNodeIds.push(blockId) + } + } + + for (const origId of origNodeIds) { + const origNode = dag.nodes.get(origId) + if (!origNode) continue + + const clonedNodeId = idMap.get(origId)! + this.cloneDAGNode(dag, origNode, clonedNodeId, clonedId, isParallel, idMap) + } + + return { startId: clonedStartId, endId: clonedEndId, idMap } + } + + /** + * Clones a single DAG node with remapped edges and updated metadata. + */ + private cloneDAGNode( + dag: DAG, + origNode: DAGNode, + clonedNodeId: string, + parentClonedId: string, + parentIsParallel: boolean, + idMap: Map + ): void { + const clonedOutgoing = new Map< + string, + { target: string; sourceHandle?: string; targetHandle?: string } + >() + for (const [, edge] of origNode.outgoingEdges) { + const clonedTarget = idMap.get(edge.target) ?? edge.target + const edgeId = edge.sourceHandle + ? `${clonedNodeId}→${clonedTarget}-${edge.sourceHandle}` + : `${clonedNodeId}→${clonedTarget}` + clonedOutgoing.set(edgeId, { + target: clonedTarget, + sourceHandle: edge.sourceHandle, + targetHandle: edge.targetHandle, + }) + } + + const clonedIncoming = new Set() + for (const incomingId of origNode.incomingEdges) { + clonedIncoming.add(idMap.get(incomingId) ?? incomingId) + } + + const metadataOverride = parentIsParallel + ? { parallelId: parentClonedId } + : { loopId: parentClonedId } + + dag.nodes.set(clonedNodeId, { + id: clonedNodeId, + block: { ...origNode.block, id: clonedNodeId }, + incomingEdges: clonedIncoming, + outgoingEdges: clonedOutgoing, + metadata: { + ...origNode.metadata, + ...metadataOverride, + ...(origNode.metadata.originalBlockId && { + originalBlockId: origNode.metadata.originalBlockId, + }), + }, + }) + } + private wireSentinelEdges( dag: DAG, parallelId: string, @@ -247,8 +507,12 @@ export class ParallelExpander { const terminalNode = dag.nodes.get(terminalNodeId) if (!terminalNode) continue - const edgeId = `${terminalNodeId}→${sentinelEndId}` - terminalNode.outgoingEdges.set(edgeId, { target: sentinelEndId }) + const handle = isLoopSentinelNodeId(terminalNodeId) ? EDGE.LOOP_EXIT : EDGE.PARALLEL_EXIT + const edgeId = `${terminalNodeId}→${sentinelEndId}-${handle}` + terminalNode.outgoingEdges.set(edgeId, { + target: sentinelEndId, + sourceHandle: handle, + }) sentinelEnd.incomingEdges.add(terminalNodeId) } } diff --git a/apps/sim/executor/utils/subflow-utils.ts b/apps/sim/executor/utils/subflow-utils.ts index 8c92f82a45..349e2f3111 100644 --- a/apps/sim/executor/utils/subflow-utils.ts +++ b/apps/sim/executor/utils/subflow-utils.ts @@ -1,11 +1,8 @@ -import { createLogger } from '@sim/logger' -import { LOOP, PARALLEL, PARSING, REFERENCE } from '@/executor/constants' +import { LOOP, PARALLEL, REFERENCE } from '@/executor/constants' import type { ContextExtensions } from '@/executor/execution/types' import { type BlockLog, type ExecutionContext, getNextExecutionOrder } from '@/executor/types' import type { VariableResolver } from '@/executor/variables/resolver' -const logger = createLogger('SubflowUtils') - const BRANCH_PATTERN = new RegExp(`${PARALLEL.BRANCH.PREFIX}\\d+${PARALLEL.BRANCH.SUFFIX}$`) const BRANCH_INDEX_PATTERN = new RegExp(`${PARALLEL.BRANCH.PREFIX}(\\d+)${PARALLEL.BRANCH.SUFFIX}$`) const LOOP_SENTINEL_START_PATTERN = new RegExp( @@ -38,12 +35,17 @@ export function buildParallelSentinelEndId(parallelId: string): string { } export function isLoopSentinelNodeId(nodeId: string): boolean { - return nodeId.includes(LOOP.SENTINEL.START_SUFFIX) || nodeId.includes(LOOP.SENTINEL.END_SUFFIX) + return ( + nodeId.startsWith(LOOP.SENTINEL.PREFIX) && + (nodeId.endsWith(LOOP.SENTINEL.START_SUFFIX) || nodeId.endsWith(LOOP.SENTINEL.END_SUFFIX)) + ) } export function isParallelSentinelNodeId(nodeId: string): boolean { return ( - nodeId.includes(PARALLEL.SENTINEL.START_SUFFIX) || nodeId.includes(PARALLEL.SENTINEL.END_SUFFIX) + nodeId.startsWith(PARALLEL.SENTINEL.PREFIX) && + (nodeId.endsWith(PARALLEL.SENTINEL.START_SUFFIX) || + nodeId.endsWith(PARALLEL.SENTINEL.END_SUFFIX)) ) } @@ -80,19 +82,82 @@ export function extractBaseBlockId(branchNodeId: string): string { export function extractBranchIndex(branchNodeId: string): number | null { const match = branchNodeId.match(BRANCH_INDEX_PATTERN) - return match ? Number.parseInt(match[1], PARSING.JSON_RADIX) : null + return match ? Number.parseInt(match[1], 10) : null } export function isBranchNodeId(nodeId: string): boolean { return BRANCH_PATTERN.test(nodeId) } -export function isLoopNode(nodeId: string): boolean { - return isLoopSentinelNodeId(nodeId) || nodeId.startsWith(LOOP.SENTINEL.PREFIX) +const OUTER_BRANCH_PATTERN = /__obranch-(\d+)/ +const OUTER_BRANCH_STRIP_PATTERN = /__obranch-\d+/g +const CLONE_SEQ_STRIP_PATTERN = /__clone\d+/g + +/** + * Extracts the outer branch index from a cloned subflow ID. + * Cloned IDs follow the pattern `{originalId}__obranch-{index}`. + * Returns undefined if the ID is not a clone. + */ +export function extractOuterBranchIndex(clonedId: string): number | undefined { + const match = clonedId.match(OUTER_BRANCH_PATTERN) + return match ? Number.parseInt(match[1], 10) : undefined } -export function isParallelNode(nodeId: string): boolean { - return isBranchNodeId(nodeId) || isParallelSentinelNodeId(nodeId) +/** + * Strips all clone suffixes (`__obranch-N`) and branch subscripts (`₍N₎`) + * from a node ID, returning the original workflow-level block ID. + */ +export function stripCloneSuffixes(nodeId: string): string { + return extractBaseBlockId( + nodeId.replace(OUTER_BRANCH_STRIP_PATTERN, '').replace(CLONE_SEQ_STRIP_PATTERN, '') + ) +} + +/** + * Builds a cloned subflow ID from an original ID and outer branch index. + */ +export function buildClonedSubflowId(originalId: string, branchIndex: number): string { + return `${originalId}__obranch-${branchIndex}` +} + +/** + * Strips outer-branch clone suffixes (`__obranch-N`) from an ID, + * returning the original workflow-level subflow ID. + */ +export function stripOuterBranchSuffix(id: string): string { + return id.replace(OUTER_BRANCH_STRIP_PATTERN, '').replace(CLONE_SEQ_STRIP_PATTERN, '') +} + +/** + * Finds the effective (possibly cloned) container ID for a subflow, + * given the current node's ID and an execution map (loopExecutions or parallelExecutions). + * + * When inside a cloned subflow (e.g., loop-1__obranch-2), the execution scope is + * stored under the cloned ID, not the original. This function extracts the `__obranch-N` + * suffix from the current node ID, constructs the candidate cloned container ID, and + * checks if it exists in the execution map. + * + * Returns the effective ID (cloned or original) that exists in the map. + */ +export function findEffectiveContainerId( + originalId: string, + currentNodeId: string, + executionMap: Map +): string { + // Prefer the cloned variant when currentNodeId carries an __obranch-N suffix. + // During concurrent parallel-in-loop execution both the original (branch 0) + // and cloned variants coexist in the map; the clone is the correct scope. + const match = currentNodeId.match(OUTER_BRANCH_PATTERN) + if (match) { + const candidateId = buildClonedSubflowId(originalId, Number.parseInt(match[1], 10)) + if (executionMap.has(candidateId)) { + return candidateId + } + } + + // Return original ID — for branch-0 (non-cloned) or when scope is missing. + // Callers handle the missing-scope case gracefully. + return originalId } export function normalizeNodeId(nodeId: string): string { diff --git a/apps/sim/executor/variables/resolvers/loop.test.ts b/apps/sim/executor/variables/resolvers/loop.test.ts index faabb48ca2..862072a516 100644 --- a/apps/sim/executor/variables/resolvers/loop.test.ts +++ b/apps/sim/executor/variables/resolvers/loop.test.ts @@ -7,33 +7,46 @@ import type { ResolutionContext } from './reference' vi.mock('@sim/logger', () => loggerMock) -/** - * Creates a minimal workflow for testing. - */ -function createTestWorkflow( - loops: Record = {} -) { - // Ensure each loop has required fields +interface LoopDef { + nodes: string[] + id?: string + iterations?: number + loopType?: 'for' | 'forEach' +} + +interface BlockDef { + id: string + name: string +} + +function createTestWorkflow(loops: Record = {}, blockDefs: BlockDef[] = []) { const normalizedLoops: Record = {} for (const [key, loop] of Object.entries(loops)) { normalizedLoops[key] = { id: loop.id ?? key, nodes: loop.nodes, iterations: loop.iterations ?? 1, + ...(loop.loopType && { loopType: loop.loopType }), } } + const blocks = blockDefs.map((b) => ({ + id: b.id, + position: { x: 0, y: 0 }, + config: { tool: 'test', params: {} }, + inputs: {}, + outputs: {}, + metadata: { id: 'function', name: b.name }, + enabled: true, + })) return { version: '1.0', - blocks: [], + blocks, connections: [], loops: normalizedLoops, parallels: {}, } } -/** - * Creates a test loop scope. - */ function createLoopScope(overrides: Partial = {}): LoopScope { return { iteration: 0, @@ -43,19 +56,19 @@ function createLoopScope(overrides: Partial = {}): LoopScope { } } -/** - * Creates a minimal ResolutionContext for testing. - */ function createTestContext( currentNodeId: string, loopScope?: LoopScope, - loopExecutions?: Map + loopExecutions?: Map, + blockOutputs?: Record ): ResolutionContext { return { executionContext: { loopExecutions: loopExecutions ?? new Map(), }, - executionState: {}, + executionState: { + getBlockOutput: (id: string) => blockOutputs?.[id], + }, currentNodeId, loopScope, } as ResolutionContext @@ -304,4 +317,127 @@ describe('LoopResolver', () => { expect(resolver.resolve('', ctx)).toBe(2) }) }) + + describe('named loop references', () => { + it.concurrent('should resolve named loop by block name', () => { + const workflow = createTestWorkflow({ 'loop-1': { nodes: ['block-1'] } }, [ + { id: 'loop-1', name: 'Loop 1' }, + ]) + const resolver = new LoopResolver(workflow) + expect(resolver.canResolve('')).toBe(true) + }) + + it.concurrent('should resolve index via named reference for block inside the loop', () => { + const workflow = createTestWorkflow({ 'loop-1': { nodes: ['block-1'] } }, [ + { id: 'loop-1', name: 'Loop 1' }, + ]) + const resolver = new LoopResolver(workflow) + const loopScope = createLoopScope({ iteration: 3 }) + const loopExecutions = new Map([['loop-1', loopScope]]) + const ctx = createTestContext('block-1', undefined, loopExecutions) + + expect(resolver.resolve('', ctx)).toBe(3) + }) + + it.concurrent('should resolve index for block in a nested descendant loop', () => { + const workflow = createTestWorkflow( + { + 'loop-outer': { nodes: ['loop-inner', 'block-a'] }, + 'loop-inner': { nodes: ['block-b'] }, + }, + [ + { id: 'loop-outer', name: 'Loop 1' }, + { id: 'loop-inner', name: 'Loop 2' }, + ] + ) + const resolver = new LoopResolver(workflow) + const outerScope = createLoopScope({ iteration: 2 }) + const innerScope = createLoopScope({ iteration: 4 }) + const loopExecutions = new Map([ + ['loop-outer', outerScope], + ['loop-inner', innerScope], + ]) + const ctx = createTestContext('block-b', undefined, loopExecutions) + + expect(resolver.resolve('', ctx)).toBe(2) + expect(resolver.resolve('', ctx)).toBe(4) + expect(resolver.resolve('', ctx)).toBe(4) + }) + + it.concurrent('should return undefined for index when block is outside the loop', () => { + const workflow = createTestWorkflow({ 'loop-1': { nodes: ['block-1'] } }, [ + { id: 'loop-1', name: 'Loop 1' }, + ]) + const resolver = new LoopResolver(workflow) + const loopScope = createLoopScope({ iteration: 3 }) + const loopExecutions = new Map([['loop-1', loopScope]]) + const ctx = createTestContext('block-outside', undefined, loopExecutions) + + expect(resolver.resolve('', ctx)).toBeUndefined() + }) + + it.concurrent('should resolve result from anywhere after loop completes', () => { + const workflow = createTestWorkflow({ 'loop-1': { nodes: ['block-1'] } }, [ + { id: 'loop-1', name: 'Loop 1' }, + ]) + const resolver = new LoopResolver(workflow) + const results = [[{ response: 'a' }], [{ response: 'b' }]] + const ctx = createTestContext('block-outside', undefined, new Map(), { + 'loop-1': { results }, + }) + + expect(resolver.resolve('', ctx)).toEqual(results) + expect(resolver.resolve('', ctx)).toEqual(results) + }) + + it.concurrent('should resolve result with nested path', () => { + const workflow = createTestWorkflow({ 'loop-1': { nodes: ['block-1'] } }, [ + { id: 'loop-1', name: 'Loop 1' }, + ]) + const resolver = new LoopResolver(workflow) + const results = [[{ response: 'a' }], [{ response: 'b' }]] + const ctx = createTestContext('block-outside', undefined, new Map(), { + 'loop-1': { results }, + }) + + expect(resolver.resolve('', ctx)).toEqual([{ response: 'a' }]) + expect(resolver.resolve('', ctx)).toBe('b') + }) + + it.concurrent('should resolve forEach properties via named reference', () => { + const workflow = createTestWorkflow( + { 'loop-1': { nodes: ['block-1'], loopType: 'forEach' } }, + [{ id: 'loop-1', name: 'Loop 1' }] + ) + const resolver = new LoopResolver(workflow) + const items = ['x', 'y', 'z'] + const loopScope = createLoopScope({ iteration: 1, item: 'y', items }) + const loopExecutions = new Map([['loop-1', loopScope]]) + const ctx = createTestContext('block-1', undefined, loopExecutions) + + expect(resolver.resolve('', ctx)).toBe(1) + expect(resolver.resolve('', ctx)).toBe('y') + expect(resolver.resolve('', ctx)).toEqual(items) + }) + + it.concurrent('should throw InvalidFieldError for unknown property on named ref', () => { + const workflow = createTestWorkflow({ 'loop-1': { nodes: ['block-1'] } }, [ + { id: 'loop-1', name: 'Loop 1' }, + ]) + const resolver = new LoopResolver(workflow) + const loopScope = createLoopScope({ iteration: 0 }) + const loopExecutions = new Map([['loop-1', loopScope]]) + const ctx = createTestContext('block-1', undefined, loopExecutions) + + expect(() => resolver.resolve('', ctx)).toThrow(InvalidFieldError) + }) + + it.concurrent('should not resolve named ref when no matching block exists', () => { + const workflow = createTestWorkflow({ 'loop-1': { nodes: ['block-1'] } }, [ + { id: 'loop-1', name: 'Loop 1' }, + ]) + const resolver = new LoopResolver(workflow) + expect(resolver.canResolve('')).toBe(false) + }) + }) }) diff --git a/apps/sim/executor/variables/resolvers/loop.ts b/apps/sim/executor/variables/resolvers/loop.ts index 3abaec58c3..8df5766882 100644 --- a/apps/sim/executor/variables/resolvers/loop.ts +++ b/apps/sim/executor/variables/resolvers/loop.ts @@ -1,7 +1,11 @@ import { createLogger } from '@sim/logger' -import { isReference, parseReferencePath, REFERENCE } from '@/executor/constants' +import { isReference, normalizeName, parseReferencePath, REFERENCE } from '@/executor/constants' import { InvalidFieldError } from '@/executor/utils/block-reference' -import { extractBaseBlockId } from '@/executor/utils/subflow-utils' +import { + findEffectiveContainerId, + stripCloneSuffixes, + stripOuterBranchSuffix, +} from '@/executor/utils/subflow-utils' import { navigatePath, type ResolutionContext, @@ -12,9 +16,19 @@ import type { SerializedWorkflow } from '@/serializer/types' const logger = createLogger('LoopResolver') export class LoopResolver implements Resolver { - constructor(private workflow: SerializedWorkflow) {} + private loopNameToId: Map - private static KNOWN_PROPERTIES = ['iteration', 'index', 'item', 'currentItem', 'items'] + constructor(private workflow: SerializedWorkflow) { + this.loopNameToId = new Map() + for (const block of workflow.blocks) { + if (workflow.loops[block.id] && block.metadata?.name) { + this.loopNameToId.set(normalizeName(block.metadata.name), block.id) + } + } + } + + private static OUTPUT_PROPERTIES = new Set(['result', 'results']) + private static KNOWN_PROPERTIES = new Set(['iteration', 'index', 'item', 'currentItem', 'items']) canResolve(reference: string): boolean { if (!isReference(reference)) { @@ -25,7 +39,7 @@ export class LoopResolver implements Resolver { return false } const [type] = parts - return type === REFERENCE.PREFIX.LOOP + return type === REFERENCE.PREFIX.LOOP || this.loopNameToId.has(type) } resolve(reference: string, context: ResolutionContext): any { @@ -35,14 +49,67 @@ export class LoopResolver implements Resolver { return undefined } - const loopId = this.findLoopForBlock(context.currentNodeId) - let loopScope = context.loopScope + const [firstPart, ...rest] = parts + const isGenericRef = firstPart === REFERENCE.PREFIX.LOOP - if (!loopScope) { - if (!loopId) { + let targetLoopId: string | undefined + + if (isGenericRef) { + targetLoopId = this.findInnermostLoopForBlock(context.currentNodeId) + if (!targetLoopId && !context.loopScope) { return undefined } - loopScope = context.executionContext.loopExecutions?.get(loopId) + } else { + targetLoopId = this.loopNameToId.get(firstPart) + if (!targetLoopId) { + return undefined + } + } + + // Resolve the effective (possibly cloned) loop ID for scope/output lookups + if (targetLoopId && context.executionContext.loopExecutions) { + targetLoopId = findEffectiveContainerId( + targetLoopId, + context.currentNodeId, + context.executionContext.loopExecutions + ) + } + + if (rest.length > 0) { + const property = rest[0] + + if (LoopResolver.OUTPUT_PROPERTIES.has(property)) { + if (!targetLoopId) { + return undefined + } + return this.resolveOutput(targetLoopId, rest.slice(1), context) + } + + if (!LoopResolver.KNOWN_PROPERTIES.has(property)) { + const isForEach = targetLoopId + ? this.isForEachLoop(targetLoopId) + : context.loopScope?.items !== undefined + const availableFields = isForEach + ? ['index', 'currentItem', 'items', 'result'] + : ['index', 'result'] + throw new InvalidFieldError(firstPart, property, availableFields) + } + + if (!isGenericRef && targetLoopId) { + if (!this.isBlockInLoopOrDescendant(context.currentNodeId, targetLoopId)) { + logger.warn('Block is not inside the referenced loop', { + reference, + blockId: context.currentNodeId, + loopId: targetLoopId, + }) + return undefined + } + } + } + + let loopScope = isGenericRef ? context.loopScope : undefined + if (!loopScope && targetLoopId) { + loopScope = context.executionContext.loopExecutions?.get(targetLoopId) } if (!loopScope) { @@ -50,26 +117,20 @@ export class LoopResolver implements Resolver { return undefined } - const isForEach = loopId ? this.isForEachLoop(loopId) : loopScope.items !== undefined - - if (parts.length === 1) { - const result: Record = { + if (rest.length === 0) { + const obj: Record = { index: loopScope.iteration, } if (loopScope.item !== undefined) { - result.currentItem = loopScope.item + obj.currentItem = loopScope.item } if (loopScope.items !== undefined) { - result.items = loopScope.items + obj.items = loopScope.items } - return result + return obj } - const [_, property, ...pathParts] = parts - if (!LoopResolver.KNOWN_PROPERTIES.includes(property)) { - const availableFields = isForEach ? ['index', 'currentItem', 'items'] : ['index'] - throw new InvalidFieldError('loop', property, availableFields) - } + const [property, ...pathParts] = rest let value: any switch (property) { @@ -93,20 +154,84 @@ export class LoopResolver implements Resolver { return value } - private findLoopForBlock(blockId: string): string | undefined { - const baseId = extractBaseBlockId(blockId) - for (const loopId of Object.keys(this.workflow.loops || {})) { - const loopConfig = this.workflow.loops[loopId] - if (loopConfig.nodes.includes(baseId)) { - return loopId + private resolveOutput(loopId: string, pathParts: string[], context: ResolutionContext): unknown { + const output = context.executionState.getBlockOutput(loopId) + if (!output || typeof output !== 'object') { + return undefined + } + const value = (output as Record).results + if (pathParts.length > 0) { + return navigatePath(value, pathParts) + } + return value + } + + private findInnermostLoopForBlock(blockId: string): string | undefined { + const baseId = stripCloneSuffixes(blockId) + const loops = this.workflow.loops || {} + const candidateLoopIds = Object.keys(loops).filter((loopId) => + loops[loopId].nodes.includes(baseId) + ) + if (candidateLoopIds.length === 0) return undefined + if (candidateLoopIds.length === 1) return candidateLoopIds[0] + + // Return the innermost: the loop that is not an ancestor of any other candidate. + // In a valid DAG, exactly one candidate will satisfy this (circular containment is impossible). + return candidateLoopIds.find((candidateId) => + candidateLoopIds.every( + (otherId) => otherId === candidateId || !loops[candidateId].nodes.includes(otherId) + ) + ) + } + + private isBlockInLoopOrDescendant(blockId: string, targetLoopId: string): boolean { + const baseId = stripCloneSuffixes(blockId) + const originalLoopId = stripOuterBranchSuffix(targetLoopId) + const targetLoop = this.workflow.loops?.[originalLoopId] + if (!targetLoop) { + return false + } + if (targetLoop.nodes.includes(baseId)) { + return true + } + const directLoopId = this.findInnermostLoopForBlock(blockId) + if (!directLoopId) { + return false + } + if (directLoopId === originalLoopId) { + return true + } + return this.isLoopNestedInside(directLoopId, originalLoopId) + } + + private isLoopNestedInside( + childLoopId: string, + ancestorLoopId: string, + visited = new Set() + ): boolean { + if (visited.has(ancestorLoopId)) return false + visited.add(ancestorLoopId) + + const ancestorLoop = this.workflow.loops?.[ancestorLoopId] + if (!ancestorLoop) { + return false + } + if (ancestorLoop.nodes.includes(childLoopId)) { + return true + } + for (const nodeId of ancestorLoop.nodes) { + if (this.workflow.loops[nodeId]) { + if (this.isLoopNestedInside(childLoopId, nodeId, visited)) { + return true + } } } - - return undefined + return false } private isForEachLoop(loopId: string): boolean { - const loopConfig = this.workflow.loops?.[loopId] + const originalId = stripOuterBranchSuffix(loopId) + const loopConfig = this.workflow.loops?.[originalId] return loopConfig?.loopType === 'forEach' } } diff --git a/apps/sim/executor/variables/resolvers/parallel.test.ts b/apps/sim/executor/variables/resolvers/parallel.test.ts index 9466fcad23..2c786638a6 100644 --- a/apps/sim/executor/variables/resolvers/parallel.test.ts +++ b/apps/sim/executor/variables/resolvers/parallel.test.ts @@ -16,19 +16,16 @@ function createTestWorkflow( nodes: string[] id?: string distribution?: any - distributionItems?: any parallelType?: 'count' | 'collection' } > = {} ) { - // Ensure each parallel has required fields const normalizedParallels: Record< string, { id: string nodes: string[] distribution?: any - distributionItems?: any parallelType?: 'count' | 'collection' } > = {} @@ -37,7 +34,6 @@ function createTestWorkflow( id: parallel.id ?? key, nodes: parallel.nodes, distribution: parallel.distribution, - distributionItems: parallel.distributionItems, parallelType: parallel.parallelType, } } @@ -366,9 +362,9 @@ describe('ParallelResolver', () => { expect(resolver.resolve('', ctx)).toEqual([]) }) - it.concurrent('should handle distributionItems property as fallback', () => { + it.concurrent('should resolve distribution items from distribution property', () => { const workflow = createTestWorkflow({ - 'parallel-1': { nodes: ['block-1'], distributionItems: ['fallback1', 'fallback2'] }, + 'parallel-1': { nodes: ['block-1'], distribution: ['fallback1', 'fallback2'] }, }) const resolver = new ParallelResolver(workflow) const ctx = createTestContext('block-1₍0₎') diff --git a/apps/sim/executor/variables/resolvers/parallel.ts b/apps/sim/executor/variables/resolvers/parallel.ts index 4481d0fe41..cc3cad1e84 100644 --- a/apps/sim/executor/variables/resolvers/parallel.ts +++ b/apps/sim/executor/variables/resolvers/parallel.ts @@ -1,20 +1,34 @@ import { createLogger } from '@sim/logger' -import { isReference, parseReferencePath, REFERENCE } from '@/executor/constants' +import { isReference, normalizeName, parseReferencePath, REFERENCE } from '@/executor/constants' import { InvalidFieldError } from '@/executor/utils/block-reference' -import { extractBaseBlockId, extractBranchIndex } from '@/executor/utils/subflow-utils' +import { + extractBranchIndex, + findEffectiveContainerId, + stripCloneSuffixes, + stripOuterBranchSuffix, +} from '@/executor/utils/subflow-utils' import { navigatePath, type ResolutionContext, type Resolver, } from '@/executor/variables/resolvers/reference' -import type { SerializedWorkflow } from '@/serializer/types' +import type { SerializedParallel, SerializedWorkflow } from '@/serializer/types' const logger = createLogger('ParallelResolver') export class ParallelResolver implements Resolver { - constructor(private workflow: SerializedWorkflow) {} + private parallelNameToId: Map - private static KNOWN_PROPERTIES = ['index', 'currentItem', 'items'] + constructor(private workflow: SerializedWorkflow) { + this.parallelNameToId = new Map() + for (const block of workflow.blocks) { + if (workflow.parallels?.[block.id] && block.metadata?.name) { + this.parallelNameToId.set(normalizeName(block.metadata.name), block.id) + } + } + } + + private static KNOWN_PROPERTIES = new Set(['index', 'currentItem', 'items']) canResolve(reference: string): boolean { if (!isReference(reference)) { @@ -25,7 +39,7 @@ export class ParallelResolver implements Resolver { return false } const [type] = parts - return type === REFERENCE.PREFIX.PARALLEL + return type === REFERENCE.PREFIX.PARALLEL || this.parallelNameToId.has(type) } resolve(reference: string, context: ResolutionContext): any { @@ -35,64 +49,85 @@ export class ParallelResolver implements Resolver { return undefined } - const parallelId = this.findParallelForBlock(context.currentNodeId) - if (!parallelId) { + const [firstPart, ...rest] = parts + const isGenericRef = firstPart === REFERENCE.PREFIX.PARALLEL + + // For named references, resolve to the specific parallel ID + let targetParallelId: string | undefined + if (isGenericRef) { + targetParallelId = this.findInnermostParallelForBlock(context.currentNodeId) + } else { + targetParallelId = this.parallelNameToId.get(firstPart) + } + + if (!targetParallelId) { return undefined } - const parallelConfig = this.workflow.parallels?.[parallelId] + // Resolve the effective (possibly cloned) parallel ID for scope lookups + if (context.executionContext.parallelExecutions) { + targetParallelId = findEffectiveContainerId( + targetParallelId, + context.currentNodeId, + context.executionContext.parallelExecutions + ) + } + + // Look up config using the original (non-cloned) ID + const originalParallelId = stripOuterBranchSuffix(targetParallelId) + const parallelConfig = this.workflow.parallels?.[originalParallelId] if (!parallelConfig) { - logger.warn('Parallel config not found', { parallelId }) + logger.warn('Parallel config not found', { parallelId: targetParallelId }) return undefined } + if (!isGenericRef) { + if (!this.isBlockInParallelOrDescendant(context.currentNodeId, originalParallelId)) { + logger.warn('Block is not inside the referenced parallel', { + reference, + blockId: context.currentNodeId, + parallelId: targetParallelId, + }) + return undefined + } + } + const branchIndex = extractBranchIndex(context.currentNodeId) if (branchIndex === null) { return undefined } - const parallelScope = context.executionContext.parallelExecutions?.get(parallelId) + const parallelScope = context.executionContext.parallelExecutions?.get(targetParallelId) const distributionItems = parallelScope?.items ?? this.getDistributionItems(parallelConfig) - if (parts.length === 1) { - const result: Record = { - index: branchIndex, - } + const currentItem = this.resolveCurrentItem(distributionItems, branchIndex) + + if (rest.length === 0) { + const result: Record = { index: branchIndex } if (distributionItems !== undefined) { result.items = distributionItems - if (Array.isArray(distributionItems)) { - result.currentItem = distributionItems[branchIndex] - } else if (typeof distributionItems === 'object' && distributionItems !== null) { - const keys = Object.keys(distributionItems) - const key = keys[branchIndex] - result.currentItem = key !== undefined ? distributionItems[key] : undefined - } + result.currentItem = currentItem } return result } - const [_, property, ...pathParts] = parts - if (!ParallelResolver.KNOWN_PROPERTIES.includes(property)) { + const property = rest[0] + const pathParts = rest.slice(1) + + if (!ParallelResolver.KNOWN_PROPERTIES.has(property)) { const isCollection = parallelConfig.parallelType === 'collection' const availableFields = isCollection ? ['index', 'currentItem', 'items'] : ['index'] - throw new InvalidFieldError('parallel', property, availableFields) + throw new InvalidFieldError(firstPart, property, availableFields) } - let value: any + let value: unknown switch (property) { case 'index': value = branchIndex break case 'currentItem': - if (Array.isArray(distributionItems)) { - value = distributionItems[branchIndex] - } else if (typeof distributionItems === 'object' && distributionItems !== null) { - const keys = Object.keys(distributionItems) - const key = keys[branchIndex] - value = key !== undefined ? distributionItems[key] : undefined - } else { - return undefined - } + value = currentItem + if (value === undefined) return undefined break case 'items': value = distributionItems @@ -106,23 +141,83 @@ export class ParallelResolver implements Resolver { return value } - private findParallelForBlock(blockId: string): string | undefined { - const baseId = extractBaseBlockId(blockId) - if (!this.workflow.parallels) { - return undefined - } - for (const parallelId of Object.keys(this.workflow.parallels)) { - const parallelConfig = this.workflow.parallels[parallelId] - if (parallelConfig?.nodes.includes(baseId)) { - return parallelId + private findInnermostParallelForBlock(blockId: string): string | undefined { + const baseId = stripCloneSuffixes(blockId) + const parallels = this.workflow.parallels + if (!parallels) return undefined + + const candidateIds = Object.keys(parallels).filter((parallelId) => + parallels[parallelId]?.nodes.includes(baseId) + ) + if (candidateIds.length === 0) return undefined + if (candidateIds.length === 1) return candidateIds[0] + + // Return the innermost: the parallel that is not an ancestor of any other candidate. + // In a valid DAG, exactly one candidate will satisfy this (circular containment is impossible). + return candidateIds.find((candidateId) => + candidateIds.every( + (otherId) => otherId === candidateId || !parallels[candidateId]?.nodes.includes(otherId) + ) + ) + } + + private isBlockInParallelOrDescendant(blockId: string, targetParallelId: string): boolean { + const baseId = stripCloneSuffixes(blockId) + const parallels = this.workflow.parallels + if (!parallels) return false + + const targetConfig = parallels[targetParallelId] + if (!targetConfig) return false + + if (targetConfig.nodes.includes(baseId)) return true + + const directParallelId = this.findInnermostParallelForBlock(blockId) + if (!directParallelId) return false + if (directParallelId === targetParallelId) return true + + return this.isParallelNestedInside(directParallelId, targetParallelId) + } + + private isParallelNestedInside( + childParallelId: string, + ancestorParallelId: string, + visited = new Set() + ): boolean { + if (visited.has(ancestorParallelId)) return false + visited.add(ancestorParallelId) + + const ancestorConfig = this.workflow.parallels?.[ancestorParallelId] + if (!ancestorConfig) return false + + if (ancestorConfig.nodes.includes(childParallelId)) return true + + for (const nodeId of ancestorConfig.nodes) { + if (this.workflow.parallels?.[nodeId]) { + if (this.isParallelNestedInside(childParallelId, nodeId, visited)) { + return true + } } } + return false + } + private resolveCurrentItem( + distributionItems: unknown[] | undefined, + branchIndex: number + ): unknown { + if (Array.isArray(distributionItems)) { + return distributionItems[branchIndex] + } + if (typeof distributionItems === 'object' && distributionItems !== null) { + const keys = Object.keys(distributionItems) + const key = keys[branchIndex] + return key !== undefined ? (distributionItems as Record)[key] : undefined + } return undefined } - private getDistributionItems(parallelConfig: any): any[] { - const rawItems = parallelConfig.distributionItems || parallelConfig.distribution || [] + private getDistributionItems(parallelConfig: SerializedParallel): unknown[] { + const rawItems = parallelConfig.distribution ?? [] // Already an array - return as-is if (Array.isArray(rawItems)) { diff --git a/apps/sim/lib/logs/execution/trace-spans/trace-spans.test.ts b/apps/sim/lib/logs/execution/trace-spans/trace-spans.test.ts index bb7d5250af..f223656c76 100644 --- a/apps/sim/lib/logs/execution/trace-spans/trace-spans.test.ts +++ b/apps/sim/lib/logs/execution/trace-spans/trace-spans.test.ts @@ -1531,3 +1531,443 @@ describe('stripCustomToolPrefix', () => { expect(stripCustomToolPrefix('regular_tool')).toBe('regular_tool') }) }) + +describe('nested subflow grouping via parentIterations', () => { + it.concurrent('parallel-in-parallel (P1 → P2 → leaf) with only leaf BlockLogs', () => { + // Sentinel blocks do NOT produce BlockLogs. Only leaf blocks have logs. + // Each leaf has parentIterations = full ancestor chain (outermost → innermost). + const result: ExecutionResult = { + success: true, + output: { content: 'done' }, + metadata: { duration: 4000, startTime: '2024-01-01T10:00:00.000Z' }, + logs: [ + // P1 iter 0, P2 iter 0 + { + blockId: 'func-1__obranch-0__obranch-0', + blockName: 'Func (iteration 0)', + blockType: 'function', + startedAt: '2024-01-01T10:00:00.000Z', + endedAt: '2024-01-01T10:00:01.000Z', + durationMs: 1000, + success: true, + parallelId: 'p2', + iterationIndex: 0, + executionOrder: 1, + parentIterations: [ + { + iterationCurrent: 0, + iterationTotal: 2, + iterationType: 'parallel', + iterationContainerId: 'p1', + }, + ], + }, + // P1 iter 0, P2 iter 1 + { + blockId: 'func-1__obranch-1__obranch-0', + blockName: 'Func (iteration 1)', + blockType: 'function', + startedAt: '2024-01-01T10:00:01.000Z', + endedAt: '2024-01-01T10:00:02.000Z', + durationMs: 1000, + success: true, + parallelId: 'p2', + iterationIndex: 1, + executionOrder: 2, + parentIterations: [ + { + iterationCurrent: 0, + iterationTotal: 2, + iterationType: 'parallel', + iterationContainerId: 'p1', + }, + ], + }, + // P1 iter 1, P2 iter 0 + { + blockId: 'func-1__obranch-0__obranch-1', + blockName: 'Func (iteration 0)', + blockType: 'function', + startedAt: '2024-01-01T10:00:02.000Z', + endedAt: '2024-01-01T10:00:03.000Z', + durationMs: 1000, + success: true, + parallelId: 'p2__obranch-1', + iterationIndex: 0, + executionOrder: 3, + parentIterations: [ + { + iterationCurrent: 1, + iterationTotal: 2, + iterationType: 'parallel', + iterationContainerId: 'p1', + }, + ], + }, + // P1 iter 1, P2 iter 1 + { + blockId: 'func-1__obranch-1__obranch-1', + blockName: 'Func (iteration 1)', + blockType: 'function', + startedAt: '2024-01-01T10:00:03.000Z', + endedAt: '2024-01-01T10:00:04.000Z', + durationMs: 1000, + success: true, + parallelId: 'p2__obranch-1', + iterationIndex: 1, + executionOrder: 4, + parentIterations: [ + { + iterationCurrent: 1, + iterationTotal: 2, + iterationType: 'parallel', + iterationContainerId: 'p1', + }, + ], + }, + ], + } + + const { traceSpans } = buildTraceSpans(result) + const workflow = traceSpans[0] + expect(workflow.name).toBe('Workflow Execution') + + // Should have one top-level parallel container (P1) + const p1 = workflow.children!.find((s) => s.type === 'parallel')! + expect(p1).toBeDefined() + expect(p1.children).toHaveLength(2) // 2 iterations of P1 + + // P1 iteration 0 → nested P2 container + const p1Iter0 = p1.children![0] + expect(p1Iter0.name).toBe('Iteration 0') + const p2InIter0 = p1Iter0.children!.find((s) => s.type === 'parallel') + expect(p2InIter0).toBeDefined() + expect(p2InIter0!.children).toHaveLength(2) // 2 iterations of P2 + + // P1 iteration 1 → nested P2 container + const p1Iter1 = p1.children![1] + expect(p1Iter1.name).toBe('Iteration 1') + const p2InIter1 = p1Iter1.children!.find((s) => s.type === 'parallel') + expect(p2InIter1).toBeDefined() + expect(p2InIter1!.children).toHaveLength(2) + + // Leaf spans inside P2 iterations + expect(p2InIter0!.children![0].children![0].name).toBe('Func') + }) + + it.concurrent('loop-in-loop nests correctly with parentIterations', () => { + // Only leaf blocks produce BlockLogs in loops too + const result: ExecutionResult = { + success: true, + output: { content: 'done' }, + metadata: { duration: 3000, startTime: '2024-01-01T10:00:00.000Z' }, + logs: [ + // Outer iter 0, inner iter 0 + { + blockId: 'agent-1', + blockName: 'Agent (iteration 0)', + blockType: 'agent', + startedAt: '2024-01-01T10:00:00.000Z', + endedAt: '2024-01-01T10:00:01.000Z', + durationMs: 1000, + success: true, + loopId: 'inner-loop', + iterationIndex: 0, + executionOrder: 1, + parentIterations: [ + { + iterationCurrent: 0, + iterationTotal: 2, + iterationType: 'loop', + iterationContainerId: 'outer-loop', + }, + ], + }, + // Outer iter 0, inner iter 1 + { + blockId: 'agent-1', + blockName: 'Agent (iteration 1)', + blockType: 'agent', + startedAt: '2024-01-01T10:00:01.000Z', + endedAt: '2024-01-01T10:00:02.000Z', + durationMs: 1000, + success: true, + loopId: 'inner-loop', + iterationIndex: 1, + executionOrder: 2, + parentIterations: [ + { + iterationCurrent: 0, + iterationTotal: 2, + iterationType: 'loop', + iterationContainerId: 'outer-loop', + }, + ], + }, + // Outer iter 1, inner iter 0 + { + blockId: 'agent-1', + blockName: 'Agent (iteration 0)', + blockType: 'agent', + startedAt: '2024-01-01T10:00:02.000Z', + endedAt: '2024-01-01T10:00:03.000Z', + durationMs: 1000, + success: true, + loopId: 'inner-loop', + iterationIndex: 0, + executionOrder: 3, + parentIterations: [ + { + iterationCurrent: 1, + iterationTotal: 2, + iterationType: 'loop', + iterationContainerId: 'outer-loop', + }, + ], + }, + ], + } + + const { traceSpans } = buildTraceSpans(result) + const workflow = traceSpans[0] + + const outerLoop = workflow.children!.find((s) => s.type === 'loop')! + expect(outerLoop).toBeDefined() + expect(outerLoop.children).toHaveLength(2) // 2 outer iterations + + // Outer iteration 0 → inner-loop container with 2 iterations + const outerIter0 = outerLoop.children![0] + const innerLoop0 = outerIter0.children!.find((s) => s.type === 'loop') + expect(innerLoop0).toBeDefined() + expect(innerLoop0!.children).toHaveLength(2) + + // Outer iteration 1 → inner-loop container with 1 iteration + const outerIter1 = outerLoop.children![1] + const innerLoop1 = outerIter1.children!.find((s) => s.type === 'loop') + expect(innerLoop1).toBeDefined() + expect(innerLoop1!.children).toHaveLength(1) + }) + + it.concurrent('3-level nesting (P1 → P2 → P3 → leaf) groups recursively', () => { + const result: ExecutionResult = { + success: true, + output: { content: 'done' }, + metadata: { duration: 2000, startTime: '2024-01-01T10:00:00.000Z' }, + logs: [ + // Leaf: parallelId=p3, parentIterations=[p1:0, p2:0] + { + blockId: 'func-1__obranch-0__obranch-0__obranch-0', + blockName: 'Func (iteration 0)', + blockType: 'function', + startedAt: '2024-01-01T10:00:00.000Z', + endedAt: '2024-01-01T10:00:01.000Z', + durationMs: 1000, + success: true, + parallelId: 'p3', + iterationIndex: 0, + executionOrder: 1, + parentIterations: [ + { + iterationCurrent: 0, + iterationTotal: 2, + iterationType: 'parallel', + iterationContainerId: 'p1', + }, + { + iterationCurrent: 0, + iterationTotal: 2, + iterationType: 'parallel', + iterationContainerId: 'p2', + }, + ], + }, + { + blockId: 'func-1__obranch-1__obranch-0__obranch-0', + blockName: 'Func (iteration 1)', + blockType: 'function', + startedAt: '2024-01-01T10:00:01.000Z', + endedAt: '2024-01-01T10:00:02.000Z', + durationMs: 1000, + success: true, + parallelId: 'p3', + iterationIndex: 1, + executionOrder: 2, + parentIterations: [ + { + iterationCurrent: 0, + iterationTotal: 2, + iterationType: 'parallel', + iterationContainerId: 'p1', + }, + { + iterationCurrent: 0, + iterationTotal: 2, + iterationType: 'parallel', + iterationContainerId: 'p2', + }, + ], + }, + ], + } + + const { traceSpans } = buildTraceSpans(result) + const workflow = traceSpans[0] + + // P1 container + const p1 = workflow.children!.find((s) => s.type === 'parallel')! + expect(p1).toBeDefined() + expect(p1.children).toHaveLength(1) // 1 iteration of P1 + + // P1 → Iteration 0 → P2 + const p1Iter0 = p1.children![0] + const p2 = p1Iter0.children!.find((s) => s.type === 'parallel') + expect(p2).toBeDefined() + expect(p2!.children).toHaveLength(1) // 1 iteration of P2 + + // P2 → Iteration 0 → P3 + const p2Iter0 = p2!.children![0] + const p3 = p2Iter0.children!.find((s) => s.type === 'parallel') + expect(p3).toBeDefined() + expect(p3!.children).toHaveLength(2) // 2 iterations of P3 + + // P3 leaf spans + expect(p3!.children![0].children![0].name).toBe('Func') + expect(p3!.children![1].children![0].name).toBe('Func') + }) + + it.concurrent('backward compatibility: spans without parentIterations group flat', () => { + const result: ExecutionResult = { + success: true, + output: { content: 'done' }, + metadata: { duration: 2000, startTime: '2024-01-01T10:00:00.000Z' }, + logs: [ + { + blockId: 'api-1__obranch-0', + blockName: 'API (iteration 0)', + blockType: 'api', + startedAt: '2024-01-01T10:00:00.000Z', + endedAt: '2024-01-01T10:00:01.000Z', + durationMs: 1000, + success: true, + parallelId: 'p1', + iterationIndex: 0, + executionOrder: 1, + }, + { + blockId: 'api-1__obranch-1', + blockName: 'API (iteration 1)', + blockType: 'api', + startedAt: '2024-01-01T10:00:01.000Z', + endedAt: '2024-01-01T10:00:02.000Z', + durationMs: 1000, + success: true, + parallelId: 'p1', + iterationIndex: 1, + executionOrder: 2, + }, + ], + } + + const { traceSpans } = buildTraceSpans(result) + const workflow = traceSpans[0] + + // Should group into a flat parallel container with 2 iterations + const parallel = workflow.children!.find((s) => s.type === 'parallel')! + expect(parallel).toBeDefined() + expect(parallel.children).toHaveLength(2) + expect(parallel.children![0].name).toBe('Iteration 0') + expect(parallel.children![1].name).toBe('Iteration 1') + // No nested containers — leaf spans are directly inside iteration + expect(parallel.children![0].children![0].name).toBe('API') + expect(parallel.children![0].children![0].type).toBe('api') + }) + + it.concurrent('mixed: flat loop + nested parallel-in-parallel in same execution', () => { + const result: ExecutionResult = { + success: true, + output: { content: 'done' }, + metadata: { duration: 5000, startTime: '2024-01-01T10:00:00.000Z' }, + logs: [ + // Flat loop iterations (no parentIterations) + { + blockId: 'agent-1', + blockName: 'Agent (iteration 0)', + blockType: 'agent', + startedAt: '2024-01-01T10:00:00.000Z', + endedAt: '2024-01-01T10:00:01.000Z', + durationMs: 1000, + success: true, + loopId: 'loop-1', + iterationIndex: 0, + executionOrder: 1, + }, + { + blockId: 'agent-1', + blockName: 'Agent (iteration 1)', + blockType: 'agent', + startedAt: '2024-01-01T10:00:01.000Z', + endedAt: '2024-01-01T10:00:02.000Z', + durationMs: 1000, + success: true, + loopId: 'loop-1', + iterationIndex: 1, + executionOrder: 2, + }, + // Nested P1 → P2 leaf (only leaf, no sentinel logs) + { + blockId: 'func-1__obranch-0__obranch-0', + blockName: 'Func (iteration 0)', + blockType: 'function', + startedAt: '2024-01-01T10:00:02.000Z', + endedAt: '2024-01-01T10:00:03.000Z', + durationMs: 1000, + success: true, + parallelId: 'p2', + iterationIndex: 0, + executionOrder: 3, + parentIterations: [ + { + iterationCurrent: 0, + iterationTotal: 2, + iterationType: 'parallel', + iterationContainerId: 'p1', + }, + ], + }, + // Non-iteration span + { + blockId: 'starter', + blockName: 'Starter', + blockType: 'starter', + startedAt: '2024-01-01T10:00:04.000Z', + endedAt: '2024-01-01T10:00:05.000Z', + durationMs: 1000, + success: true, + executionOrder: 5, + }, + ], + } + + const { traceSpans } = buildTraceSpans(result) + const workflow = traceSpans[0] + const children = workflow.children! + + const loop = children.find((s) => s.type === 'loop') + const parallel = children.find((s) => s.type === 'parallel') + const starter = children.find((s) => s.name === 'Starter') + + expect(loop).toBeDefined() + expect(parallel).toBeDefined() + expect(starter).toBeDefined() + + // Loop should have 2 flat iterations + expect(loop!.children).toHaveLength(2) + + // P1 should have 1 iteration with nested P2 + expect(parallel!.children).toHaveLength(1) + const p1Iter0 = parallel!.children![0] + const nestedP2 = p1Iter0.children!.find((s) => s.type === 'parallel') + expect(nestedP2).toBeDefined() + expect(nestedP2!.children).toHaveLength(1) + }) +}) diff --git a/apps/sim/lib/logs/execution/trace-spans/trace-spans.ts b/apps/sim/lib/logs/execution/trace-spans/trace-spans.ts index 5fd1a92af9..b7cdd3d1c4 100644 --- a/apps/sim/lib/logs/execution/trace-spans/trace-spans.ts +++ b/apps/sim/lib/logs/execution/trace-spans/trace-spans.ts @@ -2,6 +2,7 @@ import { createLogger } from '@sim/logger' import type { ToolCall, TraceSpan } from '@/lib/logs/types' import { isWorkflowBlockType, stripCustomToolPrefix } from '@/executor/constants' import type { ExecutionResult } from '@/executor/types' +import { stripCloneSuffixes } from '@/executor/utils/subflow-utils' const logger = createLogger('TraceSpans') @@ -160,6 +161,7 @@ export function buildTraceSpans(result: ExecutionResult): { ...(log.loopId && { loopId: log.loopId }), ...(log.parallelId && { parallelId: log.parallelId }), ...(log.iterationIndex !== undefined && { iterationIndex: log.iterationIndex }), + ...(log.parentIterations?.length && { parentIterations: log.parentIterations }), } if (log.output?.providerTiming) { @@ -525,292 +527,320 @@ export function buildTraceSpans(result: ExecutionResult): { return { traceSpans: groupedRootSpans, totalDuration } } +/** + * Builds a container-level TraceSpan (iteration wrapper or top-level container) + * from its source spans and resolved children. + */ +function buildContainerSpan(opts: { + id: string + name: string + type: string + sourceSpans: TraceSpan[] + children: TraceSpan[] +}): TraceSpan { + const startTimes = opts.sourceSpans.map((s) => new Date(s.startTime).getTime()) + const endTimes = opts.sourceSpans.map((s) => new Date(s.endTime).getTime()) + const earliestStart = Math.min(...startTimes) + const latestEnd = Math.max(...endTimes) + + const hasErrors = opts.sourceSpans.some((s) => s.status === 'error') + const allErrorsHandled = + hasErrors && opts.children.every((s) => s.status !== 'error' || s.errorHandled) + + return { + id: opts.id, + name: opts.name, + type: opts.type, + duration: latestEnd - earliestStart, + startTime: new Date(earliestStart).toISOString(), + endTime: new Date(latestEnd).toISOString(), + status: hasErrors ? 'error' : 'success', + ...(allErrorsHandled && { errorHandled: true }), + children: opts.children, + } +} + +/** Counter state for generating sequential container names. */ +interface ContainerNameCounters { + loopNumbers: Map + parallelNumbers: Map + loopCounter: number + parallelCounter: number +} + +/** + * Resolves a container name from normal (non-iteration) spans or assigns a sequential number. + * Strips clone suffixes so all clones of the same container share one name/number. + */ +function resolveContainerName( + containerId: string, + containerType: 'parallel' | 'loop', + normalSpans: TraceSpan[], + counters: ContainerNameCounters +): string { + const originalId = stripCloneSuffixes(containerId) + + const matchingBlock = normalSpans.find( + (s) => s.blockId === originalId && s.type === containerType + ) + if (matchingBlock?.name) return matchingBlock.name + + if (containerType === 'parallel') { + if (!counters.parallelNumbers.has(originalId)) { + counters.parallelNumbers.set(originalId, counters.parallelCounter++) + } + return `Parallel ${counters.parallelNumbers.get(originalId)}` + } + if (!counters.loopNumbers.has(originalId)) { + counters.loopNumbers.set(originalId, counters.loopCounter++) + } + return `Loop ${counters.loopNumbers.get(originalId)}` +} + +/** + * Classifies a span's immediate container ID and type from its metadata. + * Returns undefined for non-iteration spans. + */ +function classifySpanContainer( + span: TraceSpan +): { containerId: string; containerType: 'parallel' | 'loop' } | undefined { + if (span.parallelId) { + return { containerId: span.parallelId, containerType: 'parallel' } + } + if (span.loopId) { + return { containerId: span.loopId, containerType: 'loop' } + } + // Fallback: parse from blockId for legacy data + if (span.blockId?.includes('_parallel_')) { + const match = span.blockId.match(/_parallel_([^_]+)_iteration_/) + if (match) { + return { containerId: match[1], containerType: 'parallel' } + } + } + return undefined +} + +/** + * Finds the outermost container for a span. For nested spans, this is parentIterations[0]. + * For flat spans, this is the span's own immediate container. + */ +function getOutermostContainer( + span: TraceSpan +): { containerId: string; containerType: 'parallel' | 'loop' } | undefined { + if (span.parentIterations && span.parentIterations.length > 0) { + const outermost = span.parentIterations[0] + return { + containerId: outermost.iterationContainerId, + containerType: outermost.iterationType as 'parallel' | 'loop', + } + } + return classifySpanContainer(span) +} + +/** + * Builds the iteration-level hierarchy for a container, recursively nesting + * any deeper subflows. Works with both: + * - Direct spans (spans whose immediate container matches) + * - Nested spans (spans with parentIterations pointing through this container) + */ +function buildContainerChildren( + containerType: 'parallel' | 'loop', + containerId: string, + spans: TraceSpan[], + normalSpans: TraceSpan[], + counters: ContainerNameCounters +): TraceSpan[] { + const iterationType = containerType === 'parallel' ? 'parallel-iteration' : 'loop-iteration' + + // Group spans by iteration index at this level. + // Each span's iteration index at this level comes from: + // - parentIterations[0].iterationCurrent if parentIterations[0].containerId === containerId + // - span.iterationIndex if span's immediate container === containerId + const iterationGroups = new Map() + + for (const span of spans) { + let iterIdx: number | undefined + + if ( + span.parentIterations && + span.parentIterations.length > 0 && + span.parentIterations[0].iterationContainerId === containerId + ) { + iterIdx = span.parentIterations[0].iterationCurrent + } else { + // The span's immediate container is this container + iterIdx = span.iterationIndex + } + + if (iterIdx === undefined) continue + + if (!iterationGroups.has(iterIdx)) iterationGroups.set(iterIdx, []) + iterationGroups.get(iterIdx)!.push(span) + } + + const iterationChildren: TraceSpan[] = [] + const sortedIterations = Array.from(iterationGroups.entries()).sort(([a], [b]) => a - b) + + for (const [iterationIndex, iterSpans] of sortedIterations) { + // For each span in this iteration, strip one level of ancestry and determine + // whether it belongs to this container directly or to a deeper subflow + const directLeaves: TraceSpan[] = [] + const deeperSpans: TraceSpan[] = [] + + for (const span of iterSpans) { + if ( + span.parentIterations && + span.parentIterations.length > 0 && + span.parentIterations[0].iterationContainerId === containerId + ) { + // Strip the outermost parentIteration (this container level) + deeperSpans.push({ + ...span, + parentIterations: span.parentIterations.slice(1), + }) + } else { + // This span's immediate container IS this container — it's a direct leaf + directLeaves.push({ + ...span, + name: span.name.replace(/ \(iteration \d+\)$/, ''), + }) + } + } + + // Recursively group the deeper spans (they'll form nested containers) + const nestedResult = groupIterationBlocksRecursive( + [...directLeaves, ...deeperSpans], + normalSpans, + counters + ) + + iterationChildren.push( + buildContainerSpan({ + id: `${containerId}-iteration-${iterationIndex}`, + name: `Iteration ${iterationIndex}`, + type: iterationType, + sourceSpans: iterSpans, + children: nestedResult, + }) + ) + } + + return iterationChildren +} + +/** + * Core recursive algorithm for grouping iteration blocks. + * + * Handles two cases: + * 1. **Flat** (backward compat): spans have loopId/parallelId + iterationIndex but no + * parentIterations. Grouped by immediate container → iteration → leaf. + * 2. **Nested** (new): spans have parentIterations chains. The outermost ancestor in the + * chain determines the top-level container. Iteration spans are peeled one level at a + * time and recursed. + * + * Sentinel blocks (parallel/loop containers) do NOT produce BlockLogs, so there are no + * sentinel spans to anchor grouping. Containers are synthesized from the iteration data. + */ +function groupIterationBlocksRecursive( + spans: TraceSpan[], + normalSpans: TraceSpan[], + counters: ContainerNameCounters +): TraceSpan[] { + const result: TraceSpan[] = [] + const iterationSpans: TraceSpan[] = [] + const nonIterationSpans: TraceSpan[] = [] + + for (const span of spans) { + if ( + span.name.match(/^(.+) \(iteration (\d+)\)$/) || + (span.parentIterations && span.parentIterations.length > 0) + ) { + iterationSpans.push(span) + } else { + nonIterationSpans.push(span) + } + } + + // Non-iteration spans that aren't consumed container sentinels go straight to result + const nonContainerSpans = nonIterationSpans.filter( + (span) => (span.type !== 'parallel' && span.type !== 'loop') || span.status === 'error' + ) + + if (iterationSpans.length === 0) { + result.push(...nonContainerSpans) + result.sort((a, b) => new Date(a.startTime).getTime() - new Date(b.startTime).getTime()) + return result + } + + // Group iteration spans by outermost container + const containerGroups = new Map< + string, + { type: 'parallel' | 'loop'; containerId: string; containerName: string; spans: TraceSpan[] } + >() + + for (const span of iterationSpans) { + const outermost = getOutermostContainer(span) + if (!outermost) continue + + const { containerId, containerType } = outermost + const groupKey = `${containerType}_${containerId}` + + if (!containerGroups.has(groupKey)) { + const containerName = resolveContainerName(containerId, containerType, normalSpans, counters) + containerGroups.set(groupKey, { + type: containerType, + containerId, + containerName, + spans: [], + }) + } + containerGroups.get(groupKey)!.spans.push(span) + } + + // Build each container with recursive nesting + for (const [, group] of containerGroups) { + const { type, containerId, containerName, spans: containerSpans } = group + + const iterationChildren = buildContainerChildren( + type, + containerId, + containerSpans, + normalSpans, + counters + ) + + result.push( + buildContainerSpan({ + id: `${type === 'parallel' ? 'parallel' : 'loop'}-execution-${containerId}`, + name: containerName, + type, + sourceSpans: containerSpans, + children: iterationChildren, + }) + ) + } + + result.push(...nonContainerSpans) + result.sort((a, b) => new Date(a.startTime).getTime() - new Date(b.startTime).getTime()) + + return result +} + /** * Groups iteration-based blocks (parallel and loop) by organizing their iteration spans * into a hierarchical structure with proper parent-child relationships. + * Supports recursive nesting via parentIterations (e.g., parallel-in-parallel, loop-in-loop). * * @param spans - Array of root spans to process * @returns Array of spans with iteration blocks properly grouped */ function groupIterationBlocks(spans: TraceSpan[]): TraceSpan[] { - const result: TraceSpan[] = [] - const iterationSpans: TraceSpan[] = [] - const normalSpans: TraceSpan[] = [] - - spans.forEach((span) => { - const iterationMatch = span.name.match(/^(.+) \(iteration (\d+)\)$/) - if (iterationMatch) { - iterationSpans.push(span) - } else { - normalSpans.push(span) - } - }) - - // Include loop/parallel spans that have errors (e.g., validation errors that blocked execution) - // These won't have iteration children, so they should appear directly in results - const nonIterationContainerSpans = normalSpans.filter( - (span) => (span.type !== 'parallel' && span.type !== 'loop') || span.status === 'error' - ) - - if (iterationSpans.length > 0) { - const containerGroups = new Map< - string, - { - type: 'parallel' | 'loop' - containerId: string - containerName: string - spans: TraceSpan[] - } - >() - - // Track sequential numbers for loops and parallels - const loopNumbers = new Map() - const parallelNumbers = new Map() - let loopCounter = 1 - let parallelCounter = 1 - - iterationSpans.forEach((span) => { - const iterationMatch = span.name.match(/^(.+) \(iteration (\d+)\)$/) - if (iterationMatch) { - let containerType: 'parallel' | 'loop' = 'loop' - let containerId = 'unknown' - let containerName = 'Unknown' - - // Use the loopId/parallelId from the span metadata (set during execution) - if (span.parallelId) { - containerType = 'parallel' - containerId = span.parallelId - - const parallelBlock = normalSpans.find( - (s) => s.blockId === containerId && s.type === 'parallel' - ) - - // Use custom name if available, otherwise assign sequential number - if (parallelBlock?.name) { - containerName = parallelBlock.name - } else { - if (!parallelNumbers.has(containerId)) { - parallelNumbers.set(containerId, parallelCounter++) - } - containerName = `Parallel ${parallelNumbers.get(containerId)}` - } - } else if (span.loopId) { - containerType = 'loop' - containerId = span.loopId - - const loopBlock = normalSpans.find((s) => s.blockId === containerId && s.type === 'loop') - - // Use custom name if available, otherwise assign sequential number - if (loopBlock?.name) { - containerName = loopBlock.name - } else { - if (!loopNumbers.has(containerId)) { - loopNumbers.set(containerId, loopCounter++) - } - containerName = `Loop ${loopNumbers.get(containerId)}` - } - } else { - // Fallback to old logic if metadata is missing - if (span.blockId?.includes('_parallel_')) { - const parallelMatch = span.blockId.match(/_parallel_([^_]+)_iteration_/) - if (parallelMatch) { - containerType = 'parallel' - containerId = parallelMatch[1] - - const parallelBlock = normalSpans.find( - (s) => s.blockId === containerId && s.type === 'parallel' - ) - - // Use custom name if available, otherwise assign sequential number - if (parallelBlock?.name) { - containerName = parallelBlock.name - } else { - if (!parallelNumbers.has(containerId)) { - parallelNumbers.set(containerId, parallelCounter++) - } - containerName = `Parallel ${parallelNumbers.get(containerId)}` - } - } - } else { - containerType = 'loop' - // Find the first loop as fallback - const loopBlock = normalSpans.find((s) => s.type === 'loop') - if (loopBlock?.blockId) { - containerId = loopBlock.blockId - - // Use custom name if available, otherwise assign sequential number - if (loopBlock.name) { - containerName = loopBlock.name - } else { - if (!loopNumbers.has(containerId)) { - loopNumbers.set(containerId, loopCounter++) - } - containerName = `Loop ${loopNumbers.get(containerId)}` - } - } else { - containerId = 'loop-1' - containerName = 'Loop 1' - } - } - } - - const groupKey = `${containerType}_${containerId}` - - if (!containerGroups.has(groupKey)) { - containerGroups.set(groupKey, { - type: containerType, - containerId, - containerName, - spans: [], - }) - } - - containerGroups.get(groupKey)!.spans.push(span) - } - }) - - containerGroups.forEach((group, groupKey) => { - const { type, containerId, containerName, spans } = group - - const iterationGroups = new Map() - - spans.forEach((span) => { - const iterationMatch = span.name.match(/^(.+) \(iteration (\d+)\)$/) - if (iterationMatch) { - const iterationIndex = Number.parseInt(iterationMatch[2]) - - if (!iterationGroups.has(iterationIndex)) { - iterationGroups.set(iterationIndex, []) - } - iterationGroups.get(iterationIndex)!.push(span) - } - }) - - if (type === 'parallel') { - const allIterationSpans = spans - - const startTimes = allIterationSpans.map((span) => new Date(span.startTime).getTime()) - const endTimes = allIterationSpans.map((span) => new Date(span.endTime).getTime()) - const earliestStart = Math.min(...startTimes) - const latestEnd = Math.max(...endTimes) - const totalDuration = latestEnd - earliestStart - - const iterationChildren: TraceSpan[] = [] - - const sortedIterations = Array.from(iterationGroups.entries()).sort(([a], [b]) => a - b) - - sortedIterations.forEach(([iterationIndex, spans]) => { - const iterStartTimes = spans.map((span) => new Date(span.startTime).getTime()) - const iterEndTimes = spans.map((span) => new Date(span.endTime).getTime()) - const iterEarliestStart = Math.min(...iterStartTimes) - const iterLatestEnd = Math.max(...iterEndTimes) - const iterDuration = iterLatestEnd - iterEarliestStart - - const hasErrors = spans.some((span) => span.status === 'error') - const allErrorsHandled = - hasErrors && spans.every((span) => span.status !== 'error' || span.errorHandled) - - const iterationSpan: TraceSpan = { - id: `${containerId}-iteration-${iterationIndex}`, - name: `Iteration ${iterationIndex}`, - type: 'parallel-iteration', - duration: iterDuration, - startTime: new Date(iterEarliestStart).toISOString(), - endTime: new Date(iterLatestEnd).toISOString(), - status: hasErrors ? 'error' : 'success', - ...(allErrorsHandled && { errorHandled: true }), - children: spans.map((span) => ({ - ...span, - name: span.name.replace(/ \(iteration \d+\)$/, ''), - })), - } - - iterationChildren.push(iterationSpan) - }) - - const hasErrors = allIterationSpans.some((span) => span.status === 'error') - const allErrorsHandled = - hasErrors && - iterationChildren.every((span) => span.status !== 'error' || span.errorHandled) - const parallelContainer: TraceSpan = { - id: `parallel-execution-${containerId}`, - name: containerName, - type: 'parallel', - duration: totalDuration, - startTime: new Date(earliestStart).toISOString(), - endTime: new Date(latestEnd).toISOString(), - status: hasErrors ? 'error' : 'success', - ...(allErrorsHandled && { errorHandled: true }), - children: iterationChildren, - } - - result.push(parallelContainer) - } else { - const allIterationSpans = spans - - const startTimes = allIterationSpans.map((span) => new Date(span.startTime).getTime()) - const endTimes = allIterationSpans.map((span) => new Date(span.endTime).getTime()) - const earliestStart = Math.min(...startTimes) - const latestEnd = Math.max(...endTimes) - const totalDuration = latestEnd - earliestStart - - const iterationChildren: TraceSpan[] = [] - - const sortedIterations = Array.from(iterationGroups.entries()).sort(([a], [b]) => a - b) - - sortedIterations.forEach(([iterationIndex, spans]) => { - const iterStartTimes = spans.map((span) => new Date(span.startTime).getTime()) - const iterEndTimes = spans.map((span) => new Date(span.endTime).getTime()) - const iterEarliestStart = Math.min(...iterStartTimes) - const iterLatestEnd = Math.max(...iterEndTimes) - const iterDuration = iterLatestEnd - iterEarliestStart - - const hasErrors = spans.some((span) => span.status === 'error') - const allErrorsHandled = - hasErrors && spans.every((span) => span.status !== 'error' || span.errorHandled) - - const iterationSpan: TraceSpan = { - id: `${containerId}-iteration-${iterationIndex}`, - name: `Iteration ${iterationIndex}`, - type: 'loop-iteration', - duration: iterDuration, - startTime: new Date(iterEarliestStart).toISOString(), - endTime: new Date(iterLatestEnd).toISOString(), - status: hasErrors ? 'error' : 'success', - ...(allErrorsHandled && { errorHandled: true }), - children: spans.map((span) => ({ - ...span, - name: span.name.replace(/ \(iteration \d+\)$/, ''), - })), - } - - iterationChildren.push(iterationSpan) - }) - - const hasErrors = allIterationSpans.some((span) => span.status === 'error') - const allErrorsHandled = - hasErrors && - iterationChildren.every((span) => span.status !== 'error' || span.errorHandled) - const loopContainer: TraceSpan = { - id: `loop-execution-${containerId}`, - name: containerName, - type: 'loop', - duration: totalDuration, - startTime: new Date(earliestStart).toISOString(), - endTime: new Date(latestEnd).toISOString(), - status: hasErrors ? 'error' : 'success', - ...(allErrorsHandled && { errorHandled: true }), - children: iterationChildren, - } - - result.push(loopContainer) - } - }) + const normalSpans = spans.filter((s) => !s.name.match(/^(.+) \(iteration (\d+)\)$/)) + const counters: ContainerNameCounters = { + loopNumbers: new Map(), + parallelNumbers: new Map(), + loopCounter: 1, + parallelCounter: 1, } - - result.push(...nonIterationContainerSpans) - - result.sort((a, b) => new Date(a.startTime).getTime() - new Date(b.startTime).getTime()) - - return result + return groupIterationBlocksRecursive(spans, normalSpans, counters) } diff --git a/apps/sim/lib/logs/types.ts b/apps/sim/lib/logs/types.ts index 9996dbefec..eb568398fc 100644 --- a/apps/sim/lib/logs/types.ts +++ b/apps/sim/lib/logs/types.ts @@ -1,5 +1,5 @@ import type { Edge } from 'reactflow' -import type { SerializableExecutionState } from '@/executor/execution/types' +import type { ParentIteration, SerializableExecutionState } from '@/executor/execution/types' import type { BlockLog, NormalizedBlockOutput } from '@/executor/types' import type { DeploymentStatus } from '@/stores/workflows/registry/types' import type { Loop, Parallel, WorkflowState } from '@/stores/workflows/workflow/types' @@ -194,6 +194,7 @@ export interface TraceSpan { loopId?: string parallelId?: string iterationIndex?: number + parentIterations?: ParentIteration[] } export interface WorkflowExecutionSummary { diff --git a/apps/sim/lib/workflows/executor/execution-events.ts b/apps/sim/lib/workflows/executor/execution-events.ts index a9324d1087..2a2c06d401 100644 --- a/apps/sim/lib/workflows/executor/execution-events.ts +++ b/apps/sim/lib/workflows/executor/execution-events.ts @@ -1,4 +1,8 @@ -import type { ChildWorkflowContext, IterationContext } from '@/executor/execution/types' +import type { + ChildWorkflowContext, + IterationContext, + ParentIteration, +} from '@/executor/execution/types' import type { SubflowType } from '@/stores/workflows/workflow/types' export type ExecutionEventType = @@ -83,6 +87,7 @@ export interface BlockStartedEvent extends BaseExecutionEvent { iterationTotal?: number iterationType?: SubflowType iterationContainerId?: string + parentIterations?: ParentIteration[] childWorkflowBlockId?: string childWorkflowName?: string } @@ -108,6 +113,7 @@ export interface BlockCompletedEvent extends BaseExecutionEvent { iterationTotal?: number iterationType?: SubflowType iterationContainerId?: string + parentIterations?: ParentIteration[] childWorkflowBlockId?: string childWorkflowName?: string /** Per-invocation unique ID for correlating child block events with this workflow block. */ @@ -135,6 +141,7 @@ export interface BlockErrorEvent extends BaseExecutionEvent { iterationTotal?: number iterationType?: SubflowType iterationContainerId?: string + parentIterations?: ParentIteration[] childWorkflowBlockId?: string childWorkflowName?: string /** Per-invocation unique ID for correlating child block events with this workflow block. */ @@ -271,6 +278,9 @@ export function createSSECallbacks(options: SSECallbackOptions) { iterationTotal: iterationContext.iterationTotal, iterationType: iterationContext.iterationType, iterationContainerId: iterationContext.iterationContainerId, + ...(iterationContext.parentIterations?.length && { + parentIterations: iterationContext.parentIterations, + }), }), ...(childWorkflowContext && { childWorkflowBlockId: childWorkflowContext.parentBlockId, @@ -303,6 +313,9 @@ export function createSSECallbacks(options: SSECallbackOptions) { iterationTotal: iterationContext.iterationTotal, iterationType: iterationContext.iterationType, iterationContainerId: iterationContext.iterationContainerId, + ...(iterationContext.parentIterations?.length && { + parentIterations: iterationContext.parentIterations, + }), } : {} const childWorkflowData = childWorkflowContext diff --git a/apps/sim/stores/terminal/console/store.ts b/apps/sim/stores/terminal/console/store.ts index 3468e4d212..7479ca0d6c 100644 --- a/apps/sim/stores/terminal/console/store.ts +++ b/apps/sim/stores/terminal/console/store.ts @@ -427,6 +427,10 @@ export const useTerminalConsoleStore = create()( updatedEntry.iterationContainerId = update.iterationContainerId } + if (update.parentIterations !== undefined) { + updatedEntry.parentIterations = update.parentIterations + } + if (update.childWorkflowBlockId !== undefined) { updatedEntry.childWorkflowBlockId = update.childWorkflowBlockId } diff --git a/apps/sim/stores/terminal/console/types.ts b/apps/sim/stores/terminal/console/types.ts index 3fcfd6b1dc..80610cc8d8 100644 --- a/apps/sim/stores/terminal/console/types.ts +++ b/apps/sim/stores/terminal/console/types.ts @@ -1,3 +1,4 @@ +import type { ParentIteration } from '@/executor/execution/types' import type { NormalizedBlockOutput } from '@/executor/types' import type { SubflowType } from '@/stores/workflows/workflow/types' @@ -22,6 +23,7 @@ export interface ConsoleEntry { iterationTotal?: number iterationType?: SubflowType iterationContainerId?: string + parentIterations?: ParentIteration[] isRunning?: boolean isCanceled?: boolean /** ID of the workflow block in the parent execution that spawned this child block */ @@ -50,6 +52,7 @@ export interface ConsoleUpdate { iterationTotal?: number iterationType?: SubflowType iterationContainerId?: string + parentIterations?: ParentIteration[] childWorkflowBlockId?: string childWorkflowName?: string childWorkflowInstanceId?: string diff --git a/bun.lock b/bun.lock index 7a6c8997a4..5eed59d7b9 100644 --- a/bun.lock +++ b/bun.lock @@ -13,7 +13,7 @@ "glob": "13.0.0", "husky": "9.1.7", "lint-staged": "16.0.0", - "turbo": "2.8.11", + "turbo": "2.8.12", }, }, "apps/docs": { @@ -3493,19 +3493,19 @@ "tunnel-agent": ["tunnel-agent@0.6.0", "", { "dependencies": { "safe-buffer": "^5.0.1" } }, "sha512-McnNiV1l8RYeY8tBgEpuodCC1mLUdbSN+CYBL7kJsJNInOP8UjDDEwdk6Mw60vdLLrr5NHKZhMAOSrR2NZuQ+w=="], - "turbo": ["turbo@2.8.11", "", { "optionalDependencies": { "turbo-darwin-64": "2.8.11", "turbo-darwin-arm64": "2.8.11", "turbo-linux-64": "2.8.11", "turbo-linux-arm64": "2.8.11", "turbo-windows-64": "2.8.11", "turbo-windows-arm64": "2.8.11" }, "bin": { "turbo": "bin/turbo" } }, "sha512-H+rwSHHPLoyPOSoHdmI1zY0zy0GGj1Dmr7SeJW+nZiWLz2nex8EJ+fkdVabxXFMNEux+aywI4Sae8EqhmnOv4A=="], + "turbo": ["turbo@2.8.12", "", { "optionalDependencies": { "turbo-darwin-64": "2.8.12", "turbo-darwin-arm64": "2.8.12", "turbo-linux-64": "2.8.12", "turbo-linux-arm64": "2.8.12", "turbo-windows-64": "2.8.12", "turbo-windows-arm64": "2.8.12" }, "bin": { "turbo": "bin/turbo" } }, "sha512-auUAMLmi0eJhxDhQrxzvuhfEbICnVt0CTiYQYY8WyRJ5nwCDZxD0JG8bCSxT4nusI2CwJzmZAay5BfF6LmK7Hw=="], - "turbo-darwin-64": ["turbo-darwin-64@2.8.11", "", { "os": "darwin", "cpu": "x64" }, "sha512-XKaCWaz4OCt77oYYvGCIRpvYD4c/aNaKjRkUpv+e8rN3RZb+5Xsyew4yRO+gaHdMIUhQznXNXfHlhs+/p7lIhA=="], + "turbo-darwin-64": ["turbo-darwin-64@2.8.12", "", { "os": "darwin", "cpu": "x64" }, "sha512-EiHJmW2MeQQx+21x8hjMHw/uPhXt9PIxvDrxzOtyVwrXzL0tQmsxtO4qHf2l7uA+K6PUJ4+TjY1MHZDuCvWXrw=="], - "turbo-darwin-arm64": ["turbo-darwin-arm64@2.8.11", "", { "os": "darwin", "cpu": "arm64" }, "sha512-VvynLHGUNvQ9k7GZjRPSsRcK4VkioTfFb7O7liAk4nHKjEcMdls7GqxzjVWgJiKz3hWmQGaP9hRa9UUnhVWCxA=="], + "turbo-darwin-arm64": ["turbo-darwin-arm64@2.8.12", "", { "os": "darwin", "cpu": "arm64" }, "sha512-cbqqGN0vd7ly2TeuaM8k9AK9u1CABO4kBA5KPSqovTiLL3sORccn/mZzJSbvQf0EsYRfU34MgW5FotfwW3kx8Q=="], - "turbo-linux-64": ["turbo-linux-64@2.8.11", "", { "os": "linux", "cpu": "x64" }, "sha512-cbSn37dcm+EmkQ7DD0euy7xV7o2el4GAOr1XujvkAyKjjNvQ+6QIUeDgQcwAx3D17zPpDvfDMJY2dLQadWnkmQ=="], + "turbo-linux-64": ["turbo-linux-64@2.8.12", "", { "os": "linux", "cpu": "x64" }, "sha512-jXKw9j4r4q6s0goSXuKI3aKbQK2qiNeP25lGGEnq018TM6SWRW1CCpPMxyG91aCKrub7wDm/K45sGNT4ZFBcFQ=="], - "turbo-linux-arm64": ["turbo-linux-arm64@2.8.11", "", { "os": "linux", "cpu": "arm64" }, "sha512-+trymp2s2aBrhS04l6qFxcExzZ8ffndevuUB9c5RCeqsVpZeiWuGQlWNm5XjOmzoMayxRARZ5ma7yiWbGMiLqQ=="], + "turbo-linux-arm64": ["turbo-linux-arm64@2.8.12", "", { "os": "linux", "cpu": "arm64" }, "sha512-BRJCMdyXjyBoL0GYpvj9d2WNfMHwc3tKmJG5ATn2Efvil9LsiOsd/93/NxDqW0jACtHFNVOPnd/CBwXRPiRbwA=="], - "turbo-windows-64": ["turbo-windows-64@2.8.11", "", { "os": "win32", "cpu": "x64" }, "sha512-3kJjFSM4yw1n9Uzmi+XkAUgCae19l/bH6RJ442xo7mnZm0tpOjo33F+FYHoSVpIWVMd0HG0LDccyafPSdylQbA=="], + "turbo-windows-64": ["turbo-windows-64@2.8.12", "", { "os": "win32", "cpu": "x64" }, "sha512-vyFOlpFFzQFkikvSVhVkESEfzIopgs2J7J1rYvtSwSHQ4zmHxkC95Q8Kjkus8gg+8X2mZyP1GS5jirmaypGiPw=="], - "turbo-windows-arm64": ["turbo-windows-arm64@2.8.11", "", { "os": "win32", "cpu": "arm64" }, "sha512-JOM4uF2vuLsJUvibdR6X9QqdZr6BhC6Nhlrw4LKFPsXZZI/9HHLoqAiYRpE4MuzIwldCH/jVySnWXrI1SKto0g=="], + "turbo-windows-arm64": ["turbo-windows-arm64@2.8.12", "", { "os": "win32", "cpu": "arm64" }, "sha512-9nRnlw5DF0LkJClkIws1evaIF36dmmMEO84J5Uj4oQ8C0QTHwlH7DNe5Kq2Jdmu8GXESCNDNuUYG8Cx6W/vm3g=="], "tweetnacl": ["tweetnacl@0.14.5", "", {}, "sha512-KXXFFdAbFXY4geFIwoyNK+f5Z1b7swfXABfL7HXCmoIWMKU3dmS26672A4EeQtDzLKy7SXmfBu51JolvEKwtGA=="], diff --git a/package.json b/package.json index d4a932a341..22ede044fc 100644 --- a/package.json +++ b/package.json @@ -42,7 +42,7 @@ "glob": "13.0.0", "husky": "9.1.7", "lint-staged": "16.0.0", - "turbo": "2.8.11" + "turbo": "2.8.12" }, "lint-staged": { "*.{js,jsx,ts,tsx,json,css,scss}": [