Execution engine refactoring

[JPPhoto]

Summary

This is a backend execution-engine refactor. The goal is to make workflow execution easier to extend and maintain by giving each control-flow type clear ownership of how it schedules its next work, without changing the frontend/backend contract.

• Extracts scheduling, runtime, materialization, control-flow, and child-execution responsibilities into focused modules under invokeai/app/services/shared/.
• Routes bounded If, For, Iterate/Collect, nested control flow, and saved-workflow execution through generic execution paths.
• Preserves compatibility adapters for unsupported, forced-compatibility, and legacy snapshots.
• Updates execution-engine documentation and regression coverage.
• No files under invokeai/frontend/... changed.

This incorporates the linear For scheduling requirements from #225:

• Transfers the remaining For collection between iterations without quadratic deep-copying.
• Tracks pending prepared nodes incrementally instead of rescanning them.
• Adds structural regression coverage for both For and Iterate.

The implementation is adapted into this branch rather than being an exact cherry-pick of PR 225.

Deferred work:

I deferred removing the old control-flow-specific scheduling and materialization paths that are still needed for unsupported graphs, forced compatibility mode, and legacy snapshots.

Each removal must first prove that the generic execution path produces identical behavior, persistence, recovery, and queue lifecycle results. The frontend contract must remain unchanged, and every deletion requires focused tests and adversarial review.

This PR is already large, and I decided to hit the brakes on more changes.

Related Issues / Discussions

Closes #225: linear For-loop scheduling.

QA Instructions

Passed at the time of writing:

• 1,085 focused execution tests, 2 expected xfails.
• 15 child-execution tests.
• 4 scheduler-performance tests.
• Ruff check and format.
• Invocation-version validation.
• Documentation generation, redirects, and build.
• Whitespace validation.
• Final adversarial code reviews.

Contract checks:

• No invokeai/frontend/... changes.
• Internal execution ledgers remain persistence-only.
• Generated backend OpenAPI contains no internal ledger schemas or path references.

Manual testing:

Load existing workflows with For, Call Saved Workflow, and Iterate nodes and make sure they execute as before. Then, create similar ones from scratch and see that behavior is preserved.

Merge Plan

Checklist

• The PR has a short but descriptive title, suitable for a changelog
• Tests added / updated (if applicable)
• ❗Changes to a redux slice have a corresponding migration
• Documentation added / updated (if applicable)
• Updated What's New copy (if doing a release after this PR)

[Pfannkuchensack]

Review: PR #217 "Execution engine refactoring" (head 1ce2ffc, base bdfa4f4)

Two High findings: graphs that main runs to completion now stall or crash when an If sits inside a For/Iterate body. I would not merge until those are fixed. The nested-If and Collect-order findings (3 and 4) change behaviour and need either a fix or a documented decision.

Findings

High

1. An If with a constant condition inside a loop body stalls the session with no error

• Path: invokeai/app/services/shared/graph_materializer.py:1422-1423, also :1037-1040 and :1463-1488.
• Trigger:
  • Range -> Iterate (or For/ForReturn).
  • loop.item -> nodes a and b.
  • a -> if.true_input, b -> if.false_input.
  • condition is set on the node, not connected by an edge.
  • if.value -> Collect (or ForReturn).
• Evidence chain:
  1. The If has no condition edges, so _get_if_condition_iteration_mappings returns iter([[]]).
  2. The If is materialized once at path () with no input edges, instead of once per iteration, and goes into _pending_if_exec_nodes.
  3. _attach_pending_if_inputs never finds branch candidates at (), so a and b are never materialized.
  4. next() returns None while is_complete() is False and errors is empty.
  5. The runner loop breaks. _on_after_run_session only finalizes complete sessions (invokeai/app/services/session_processor/session_processor_default.py:537). run_queue_item returns for top-level items (invokeai/app/services/session_processor/workflow_call_runtime.py:394-395).
  6. Result: the queue item stays in_progress with no output and no error.
• Reproduced on both trees with the same venv, for condition True and False:
  • Base Iterate: complete=True, order [range, loop x3, a x3, if x3, collect], out [0, 1, 2].
  • PR Iterate: complete=False, errors={}, order [range, loop, loop, loop].
  • Base For: completes with out [0, 1, 2].
  • PR For: complete=False, order [range, loop].
• To expose this issue, add a test that runs the Iterate and For variants of this graph through GraphExecutionState, for condition True and False, and asserts is_complete() and the collected output [0, 1, 2].

2. An If whose condition comes from outside the loop, with branches inside the body, crashes with KeyError: 'loop'

• Path: invokeai/app/services/shared/graph_materializer.py:1319, reached through :1652 -> :1405 -> :1424.
• Trigger: the same body as finding 1, but a node outside the loop (for example a Boolean primitive) feeds if.condition. This is a common UI pattern: one toggle controls every iteration.
• Evidence chain:
  1. prepare() decides If readiness from the condition edges only (:1636-1641, :1649). Once cond has executed, the If is considered before loop is prepared.
  2. _has_admitted_source_mapping (:1652) calls _get_parent_iteration_mappings.
  3. That finds loop through the branch ancestry and indexes source_prepared_mapping['loop'], which does not exist yet.
  4. The ancestor-iterator guard at :1657-1661 comes later in the same and chain, so it never short-circuits.
  5. The exception escapes next(), and the processor fails the queue item with KeyError: 'loop'.
• Reproduced: base completes both For and Iterate with out [0, 1, 2]; the PR raises KeyError: 'loop' for all three branch shapes tried. Two of those shapes (a single branch node, and loop.item wired directly into the If) already fail to complete on base, but only the two-branch shape is a regression.
• Why the tests miss it: the only mixed If+Iterate differential case wires the condition from iterate.item.
• To expose this issue, add a test that runs the outside-condition variant for For and for Iterate and asserts completion and output [0, 1, 2].

Medium

3. Nested If: a side consumer of the middle If is silently skipped

• Path: invokeai/app/services/shared/graph_if_dependencies.py:238 (_get_fresh_if_branch_sources, :236-245). Admission is at :186-210, special case :192-193.
• Evidence chain:
  1. Leaf consumers of an inner If are added to the outer If's branch.
  2. In the admitted 3-If chain, the inner chain counts as inactive (invokeai/app/services/shared/graph_if_runtime.py:39-65).
  3. It is discarded (invokeai/app/services/shared/graph_scheduler.py:444-457).
  4. Main only skipped nodes whose outputs all flow into the unselected branch, so it ran the side consumer.
• Trigger: outer condition False, middle and inner True, and the middle If's output also feeds a leaf such as Save Image.
• Reproduced:
  • Main: [..., inner_if, middle_if, sink, middle_side_consumer].
  • PR: [outer_condition, outer_false, outer_if, sink]; the consumer never runs.
• Inconsistent at the router boundary: the same fan-out in a 2-If chain is not admitted, falls back, and still runs the consumer. Adding one more If changes which nodes execute.
• Test impact: the differential test at :7328-7359 asserts the new skip. The forced-compatibility run cannot catch it, because the dependency compiler is chosen by graph shape, not by scheduler.
• To expose this issue, add a test that runs the 3-If middle-leaf fan-out with outer condition False and asserts middle_side_consumer executes, as it does on main and in the 2-If case.

4. Collect output order changes for admitted fan-in shapes and differs at the admission boundary

• Path: invokeai/app/services/shared/graph_execution_runtime.py:108-128 and :153-170; planner sort at invokeai/app/services/shared/graph_iterate_planner.py:76 and :160.
• Evidence chain:
  1. Admitted shapes (2-3 direct branches, or exactly 2 body-mediated branches) sort Collect inputs by source node id, then iteration path.
  2. Main sorted by iteration path, then exec id, which interleaves branches by index.
  3. Shapes with 4 or more branches keep main's interleaving, with a random tie-break.
• Reproduced:
  • Direct fan-in [0,1] + [10,11,12]: main [0,10,1,11,12], PR [0,1,10,11,12].
  • Body fan-in with sources zz=[1,2], aa=[50]: main [1,50,2], PR [50,1,2].
• Impact: source node ids come from the UI, so users see an arbitrary branch order. Adding a fourth branch switches the ordering rule. The change is documented in invokeai/app/services/shared/README.md:53-56, but there is no release note and no test against main.
• To expose this issue, add a test that runs 2-branch and 4-branch direct fan-in with the same values and asserts one consistent ordering rule.

5. Persisted sessions are much larger and slower to load, and webv2 queue listings load every row

• Path: invokeai/app/services/shared/execution_state_migration.py:26-47 (_append_runtime_fields). Consumers are invokeai/app/services/session_queue/session_queue_sqlite.py:1870 (list_all_queue_items) and :1830, plus items_by_ids.

• Evidence chain:

  1. Every snapshot now carries four ledgers.
  2. execution_tokens deep-copies every output port and is about 54% of the payload.
  3. Every row goes through json.loads, model_validate and the full model_post_init rebuild (invokeai/app/services/shared/graph.py:3330).
  4. webv2 uses list_all and items_by_ids, and queue history is unbounded by default.

• Measured, main vs PR, same completed graphs:

  Graph	Main size	Main load	PR size	PR load
  For, 100 iterations	189 KB	3.3 ms	974 KB	49.5 ms
  For, 500 iterations	941 KB	82 ms	4.87 MB	423 ms
  Iterate, 500 items	736 KB	11 ms	2.48 MB	145 ms

• Growth is linear, not quadratic. The public response does not grow, because the ledger fields are exclude=True.

• To expose this issue, add a test that runs a 100-iteration For graph to completion and asserts a size and load-time budget on dump_execution_state relative to model_dump_json.

6. Per-item scheduling is about 1.4x to 3x slower, and the scaling test is narrower

• Path: invokeai/app/services/shared/graph.py:2615-2710; apply() runs on every complete().

• Measured, trivial body, ms per item:

  Loop	n	Base	PR
  Iterate	500	0.19	0.56
  Iterate	4000	0.31	0.90
  For	500	0.63	1.75
  For	4000	1.41	2.02

• Contributors on every completion:

  • A JSON dump of each effect (:2243).
  • A copydeep of the effects (:2668).
  • A deep copy of every output port into execution_tokens (:2528-2544).
  • Seven planner-eligibility checks, run twice (invokeai/app/services/shared/graph_scheduler.py:638-646, :665-680).

• Test coverage got narrower: the non-slow CPU-time ratio test test_loop_scheduler_overhead_is_linear was replaced by a traversal counter on results and execution_effects only. It does not cover:

  • execution_tokens.values() in invokeai/app/services/shared/graph_if_runtime.py:137-149;
  • the whole-plan scan in invokeai/app/services/shared/graph_scheduler.py:453-457;
  • _if_exec_ids per completion (:546-552);
  • any If-in-loop graph, which findings 1 and 2 currently block.

• A quadratic regression in any of those structures would pass CI. The PR's efficiency claim holds for For scaling but not for the per-item constant.

• To expose this issue, add a scaling test with an If inside the loop body that instruments execution_tokens and the scheduler plan.

Low

7. The quarantine catch misses non-ValueError load errors, so a single bad row stalls the queue (not a regression)

• Path: invokeai/app/services/session_queue/session_queue_sqlite.py:618-624.
• Evidence chain:
  1. _hydrate_queue_item only catches (TypeError, ValueError).
  2. A snapshot whose prepared_source_mapping names a missing source raises networkx.exception.NetworkXError.
  3. That escapes even with quarantine=True.
  4. In dequeue() (:583-601), the processor logs the error, sleeps one poll interval and selects the same pending row again (invokeai/app/services/session_processor/session_processor_default.py:1164-1176).
  5. The queue is blocked for all users, and list_all returns 500.
• A main-era nested-If snapshot raises a raw KeyError the same way (invokeai/app/services/shared/graph.py:3084, :3157).
• Main behaves identically, so this is not a regression, but the PR's claim that malformed snapshots are quarantined is not met.
• To expose this issue, add a test that adds the missing-source prepared_source_mapping payload to the FIFO quarantine test parameters and asserts later work still dequeues.

8. apply() rollback clears journaled _pending_if_exec_nodes, so a retried completion stalls

• Path: invokeai/app/services/shared/graph.py:446, called from the rollback at :2703-2707.
• Evidence chain:
  1. transaction.rollback() correctly restores the set.
  2. _reset_apply_derived_caches() then resets it to set().
  3. _rehydrate_ready_queues() does not rebuild it.
• Reproduced by injecting an exception into _build_execution_tokens: the retry ends with complete=False, errors={}, out=[] instead of ['A'].
• Scope: only direct complete()/apply() callers that retry; the session runner fails the session on that exception anyway.
• To expose this issue, add a test that forces an exception after mutation in apply(), retries the completion, and asserts the If graph still completes.

9. The new facade shims turn the usual wrap/spy pattern into infinite recursion

• Path: invokeai/app/services/shared/graph_validation.py:87; the same shim is in 11 helpers (:133, :152, ..., :379).
• Evidence chain:
  1. Each helper forwards to whatever name is set on the graph module.
  2. A wrapper that saved the original and assigned itself to graph.are_connection_types_compatible calls the original.
  3. The original forwards back to the wrapper.
• Reproduced: PR RecursionError after 147 calls, base OK calls 1.
• Affected: tests, extensions and community nodes that use the wrap pattern or mock.patch(wraps=...).
• To expose this issue, add a test that wraps graph.are_connection_types_compatible with a delegating spy and asserts Graph.add_edge succeeds and the spy runs exactly once.

10. Every Call Saved Workflow call keeps a full copy of the workflow in the parent session, never pruned

• Path: invokeai/app/services/shared/graph.py:2699, together with invokeai/app/invocations/call_saved_workflow.py:90.
• Evidence chain:
  1. SpawnExecutionEffect.graph holds the whole workflow JSON.
  2. It is stored in execution_effects on first apply and stored again on resume.
  3. It is serialized on every parent save.
  4. Main cleared its child state on completion (end_waiting_on_workflow_call).
• Measured: a 2.5 KB workflow makes the parent session grow from 835 to 9482 bytes. Real saved workflows are typically 50-300 KB.
• To expose this issue, add a test that runs a parent with two call nodes through resume and asserts the persisted parent holds no spawn graph afterwards.

11. Call Saved Workflow failures lose their exception type and traceback

• Path: invokeai/app/invocations/call_saved_workflow.py:102-105, reported at invokeai/app/services/session_processor/session_processor_default.py:344-352.
• Evidence chain:
  1. invoke catches every exception and records execution.fail(str(e)).
  2. The runner reports error_type="ValueError" with the message as the traceback.
  3. An exception with an empty message makes FailEffect.message (min_length=1, invokeai/app/services/shared/execution_effects.py:337) raise a pydantic ValidationError that hides the original error.
• To expose this issue, add a test that makes the spawn raise KeyError() and asserts the failed item's error_type and traceback.

Open Questions

• Stale parent write: set_queue_item_session (invokeai/app/services/session_queue/session_queue_sqlite.py:1491) writes unconditionally in _on_after_run_session. Can a child that completes on another worker in that window have its recorded completion overwritten, leaving the parent in_progress forever? The pattern already existed on main and was not reproduced with the dummy queue.
• Placeholder read as complete: get_queue_item now returns an empty-graph placeholder for unreadable snapshots, and an empty GraphExecutionState.is_complete() returns True. If a just-saved snapshot fails to reload in _on_after_run_session, is the item marked completed with no work done? No trigger was found.
• Retry of unreadable items: retry is silently skipped (invokeai/app/services/session_queue/session_queue_sqlite.py:2152), even when only the runtime ledger is bad and the graph would be retryable. The user gets no reason.
• Serialization strictness: warnings="error" in _validate_json_serializable (invokeai/app/services/shared/execution_effects.py:28) now also runs on the compatibility path for Iterate items and ForReturn payloads. Could values that main accepted with serializer warnings now fail the node?
• executed set: _mark_completed_sources (invokeai/app/services/shared/graph.py:3476-3482) no longer adds skipped sources to executed. webv2 does not read it; does legacy web or any API consumer?
• Event order: execution order changed in several If and fan-in graphs without changing results, so the order of progress events changes too.
• Removed names: is_call_saved_workflow_dynamic_input and load_all_modules are no longer exported from invokeai/app/services/shared/graph.py. Nothing in the repo uses them; community node packs are unknown.

[JPPhoto]

@Pfannkuchensack Thanks for reviewing. Findings 1 and 2 are confirmed merge blockers.

High findings

1. graph_materializer.py:1418-1424
   Confirmed. An If with a constant condition inside For or Iterate is materialized at the wrong frame, causing the session to stall with no error.

2. graph_materializer.py:1318-1321
   Confirmed. An If conditioned by a node outside the loop can access loop mappings before they exist, causing a KeyError or stall.

Medium findings

3. graph_if_dependencies.py:227-260
   Confirmed. The admitted three-If optimization can skip a valid middle-If side consumer. The documentation currently claims this topology is supported.

4. graph_execution_runtime.py:203-207
   Confirmed. Some admitted fan-in graphs now produce source-ordered Collect results instead of iteration-interleaved results. This is observable workflow behavior and requires an explicit compatibility decision or correction.

5. execution_state_migration.py:26-47
   Confirmed. Persisted snapshots are substantially larger because they retain execution ledgers. These fields remain excluded from public API responses, but persistence and loading costs need acceptance or mitigation.

6. test_for_scheduler_performance.py:78-108
   Confirmed test gap. The scaling test covers only results and execution_effects; it does not cover token scans, plan scans, or If-inside-loop scheduling.

Low findings

7. session_queue_sqlite.py:618-624
   Confirmed behavior, but not introduced by this change. NetworkXError is not quarantined, and the same behavior exists on the prior implementation.

8. graph.py:446
   Confirmed. Rollback restores _pending_if_exec_nodes, then cache reset clears it. A direct caller retrying apply() can therefore stall.

9. graph_validation.py:87-96
   Confirmed. The facade override mechanism recurses when used with the normal save-original-then-wrap spy pattern.

10. execution_effects.py:305-318
    Confirmed. Completed workflow calls retain the complete child workflow graph in the parent execution effects.

11. call_saved_workflow.py:102-105
    Confirmed. Workflow-call failures lose their original exception type and traceback. Empty exception messages can also produce a secondary FailEffect validation error.

Open questions

• Stale parent writes are theoretically possible at session_processor_default.py:533, but this pattern predates the change and was not reproduced.
• Placeholder sessions could theoretically appear complete, but no normal trigger was found after a successful save. Checking _snapshot_readable defensively would be safer.
• Unreadable retries are intentionally skipped at session_queue_sqlite.py:2152. The behavior protects the original snapshot but provides no user-facing reason.
• No valid in-tree output was found that fails because of the stricter JSON validation. Non-JSON For and ForReturn compatibility paths explicitly fall back.
• No in-repository consumer of the changed executed semantics was found outside tests and schema generation. The field remains publicly exposed, so external consumers require consideration.
• Event ordering changes are confirmed as part of the fan-in finding and can affect progress events.
• No in-repository use of the removed graph.py exports was found. External community-node compatibility cannot be proven.
• No invokeai/frontend/... files are changed. Internal execution ledgers are excluded from public schemas and client responses.