Skip to content

fix(workflows): preserve core_metadata recall fields across workflow roundtrip - #9162

Merged
lstein merged 13 commits into
invoke-ai:mainfrom
Pfannkuchensack:fix/workflow-metadata-loras-roundtrip
Jul 27, 2026
Merged

lstein merged 13 commits into
invoke-ai:mainfrom
Pfannkuchensack:fix/workflow-metadata-loras-roundtrip

Conversation

@Pfannkuchensack

@Pfannkuchensack Pfannkuchensack commented May 12, 2026

Copy link
Copy Markdown
Member

Summary

The core_metadata node is configured with pydantic extra='allow' and carries recall metadata (loras, controlnets, ipAdapters, t2iAdapters, plus model-specific extras like z_image_seed_variance_*, dype_preset, ref_images). When a generated image's graph was loaded into the workflow editor and saved, those values were being dropped:

  • The four *MetadataField collection types were not registered as StatefulFieldType, so their inputs fell through to zStatelessFieldInputInstance, whose value is z.undefined().catch(undefined) — silently coercing the array away.
  • Extra keys not declared in the OpenAPI schema were dropped earlier still, in graphToWorkflow, because template.inputs[key] was undefined and the field was skipped with a warning.

Result: the regenerated image had no recall metadata and "Recall all parameters" reported no LoRA / no variance settings — even though the backend executed the workflow correctly (lora_selector and the z_image_seed_variance_enhancer node were intact via their edges).

This change:

  • Registers LoRAMetadataField, ControlNetMetadataField, IPAdapterMetadataField, T2IAdapterMetadataField as stateful field types with passthrough zod values.
  • Adds a synthetic MetadataExtraField so undeclared keys on extra='allow' nodes round-trip through the workflow editor.
  • graphToWorkflow synthesizes an extra-field template for keys not in the node template, scoped to nodes that accept extras.
  • buildNodesGraph forwards extra values verbatim when running the workflow.
  • fieldValidators and InputFieldGate no longer treat undeclared inputs on extra-accepting nodes as errors / unexpected fields.

Adds regression tests covering the LoRA roundtrip, the extras roundtrip, and the parseSchema template type for core_metadata.loras.

Related Issues / Discussions

Fixes #9151

QA Instructions

Tested end-to-end against a real Invoke instance (v6.12.0, Z-Image Turbo) with a real LoRA. Reproduction steps for reviewers:

  1. Generate a baseline image in the Generate tab using Z-Image Turbo with at least one LoRA enabled. For extra coverage, enable Seed Variance (strength > 0).
  2. From the gallery, open the image's context menu and choose Load Workflow (or "Edit Workflow"). The workflow editor should open without "Unexpected field" errors and the Invoke button should not be blocked.
  3. Click Invoke to regenerate from the loaded workflow.
  4. Inspect the new image's metadata (Image Info / Metadata panel):
    • loras array is present, with the same model and weight as the original.
    • If Seed Variance was used: z_image_seed_variance_enabled, z_image_seed_variance_strength, z_image_seed_variance_randomize_percent are present.
  5. Click Recall all parameters on the new image — the LoRA and Variance settings should populate the side panel.
  6. Compare the two backend graphs (via the image's "Graph" debug info if available): node set and edges should be identical between the original and the workflow-driven run.

Automated coverage (added in this PR):

  • loraMetadataRoundtrip.test.tszFieldInputInstance preserves values for all four *MetadataField types and accepts null.
  • graphToWorkflow.test.ts — full graph → workflow → parseAndMigrateWorkflow roundtrip preserves both core_metadata.loras and undeclared extras (z_image_seed_variance_*).
  • parseSchemaCoreMetadata.test.tsparseSchema produces a stateful LoRAMetadataField template for core_metadata.loras (not a stateless fallback).

Run with:

cd invokeai/frontend/web
pnpm test:no-watch

Merge Plan

Frontend-only change, no migrations and no backend schema impact. Safe to merge whenever; consider including in a patch release alongside other Z-Image bug fixes since the user-visible symptom (LoRA appears to be ignored on workflow re-runs) is a regression hot spot.

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Tests added / updated (if applicable)
  • ❗Changes to a redux slice have a corresponding migration — N/A (no slice changes; persisted workflow shape is unchanged, only the zod validators are loosened to preserve previously-stripped values)
  • Documentation added / updated (if applicable) — N/A
  • Updated What's New copy (if doing a release after this PR) — defer to release author

…roundtrip

The `core_metadata` node is configured with pydantic `extra='allow'` and
carries recall metadata (`loras`, `controlnets`, `ipAdapters`,
`t2iAdapters`, plus model-specific extras like `z_image_seed_variance_*`,
`dype_preset`, `ref_images`). When a generated image's graph was loaded
into the workflow editor and saved, those values were being dropped:

  * The four `*MetadataField` collection types were not registered as
    StatefulFieldType, so their inputs fell through to
    `zStatelessFieldInputInstance`, whose `value` is
    `z.undefined().catch(undefined)` — silently coercing the array away.
  * Extra keys not declared in the OpenAPI schema were dropped earlier
    still, in `graphToWorkflow`, because `template.inputs[key]` was
    undefined and the field was skipped with a warning.

Result: the regenerated image had no recall metadata and "Recall all
parameters" reported no LoRA / no variance settings — even though the
backend executed the workflow correctly (`lora_selector` and the
`z_image_seed_variance_enhancer` node were intact via their edges).

This change:

  * Registers `LoRAMetadataField`, `ControlNetMetadataField`,
    `IPAdapterMetadataField`, `T2IAdapterMetadataField` as stateful field
    types with passthrough zod values.
  * Adds a synthetic `MetadataExtraField` so undeclared keys on
    `extra='allow'` nodes round-trip through the workflow editor.
  * `graphToWorkflow` synthesizes an extra-field template for keys not in
    the node template, scoped to nodes that accept extras.
  * `buildNodesGraph` forwards extra values verbatim when running the
    workflow.
  * `fieldValidators` and `InputFieldGate` no longer treat undeclared
    inputs on extra-accepting nodes as errors / unexpected fields.

Adds regression tests covering the LoRA roundtrip, the extras roundtrip,
and the parseSchema template type for `core_metadata.loras`.

Fixes invoke-ai#9151
@github-actions github-actions Bot added the frontend PRs that change frontend files label May 12, 2026
@lstein lstein added the 6.14.0 label Jun 3, 2026
@lstein lstein moved this to 6.14.x Theme: USER EXPERIENCE in Invoke - Community Roadmap Jun 3, 2026

@JPPhoto JPPhoto left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • invokeai/frontend/web/src/features/nodes/types/field.ts:1334

    MetadataExtraField uses z.any() and is added to the global zFieldInputInstance union. Workflow input instances do not include the field template/type when parsed; they are parsed as z.record(z.string(), zFieldInputInstance) at invokeai/frontend/web/src/features/nodes/types/invocation.ts:34. Because of that, this new catch-all input instance can match any malformed input value anywhere in any workflow, not just core_metadata extras.

    Before this PR, a stateless connection-only input with an accidental saved value would fall through to zStatelessFieldValue, which coerces to undefined at invokeai/frontend/web/src/features/nodes/types/field.ts:1362. After this PR, that same value can be preserved by the MetadataExtraField branch. Then buildNodesGraph serializes input.value for every known template at invokeai/frontend/web/src/features/nodes/util/graph/buildNodesGraph.ts:75. This lets stale or malformed workflow data become backend graph input data for fields that are supposed to be connection-only.

    Load or import a workflow with a non-core_metadata node whose stateless/connection-only input has a saved value. The parser accepts the value, and enqueue/export can send it into the backend graph.

    To expose this issue, add a test that parses a workflow containing a non-core_metadata stateless input with a value, then builds the graph and asserts that value is not serialized unless the node type explicitly accepts extras.

  • invokeai/frontend/web/src/features/nodes/util/workflow/validateWorkflow.ts:102

    The PR exempts core_metadata extras in graphToWorkflow, fieldValidators, InputFieldGate, and buildNodesGraph, but not in validateWorkflow. validateWorkflow still warns whenever an input lacks a template. That means the very extras this PR preserves, such as z_image_seed_variance_enabled, become workflow-load warnings.

    Loading from image uses graphToWorkflow when an image has a graph but no workflow at invokeai/frontend/web/src/features/workflowLibrary/hooks/useLoadWorkflowFromImage.ts:35. It then calls validateWorkflow through invokeai/frontend/web/src/features/workflowLibrary/hooks/useValidateAndLoadWorkflow.ts:54. For each synthesized core_metadata extra, template.inputs[input.name] is missing, so validation pushes nodes.missingFieldTemplate at invokeai/frontend/web/src/features/nodes/util/workflow/validateWorkflow.ts:103. The caller then shows "loaded with warnings" at invokeai/frontend/web/src/features/workflowLibrary/hooks/useValidateAndLoadWorkflow.ts:88.

    Recall/load an image whose saved graph contains core_metadata extras. The workflow loads, but the user gets a warning for intentionally valid metadata fields.

    To expose this issue, add a validateWorkflow test using a core_metadata workflow node with an undeclared extra and assert validation returns no warnings while preserving the input.

  • invokeai/frontend/web/src/features/nodes/util/node/nodeUpdate.ts:61: updateNode still trims node inputs to keys(defaults.data.inputs). If core_metadata ever gets a compatible template version bump, update can drop the preserved extra inputs. This issue depends on a future version mismatch, but it is the same preservation gap in another workflow path.

  • invokeai/frontend/web/src/features/nodes/util/workflow/graphToWorkflow.test.ts: the tests use proxy templates whose inputs appear to contain any requested key. This is useful for graph breadth, but it hides real missing-template behavior. A direct test with ordinary object templates would better cover the synthesized-extra path.

  • The new metadata value schemas intentionally model metadata as opaque object arrays or arbitrary values. That matches the round-trip goal, but it makes the parser less capable of catching malformed metadata. The key safety boundary should be scoping: arbitrary extras should be accepted only after the node type is known to accept them.

JPPhoto and others added 3 commits July 1, 2026 03:57
…nodes

The MetadataExtraField input instance (value: z.any()) was added to the
global field-instance union, so it matched any malformed input value on
any node during workflow parsing (inputs are parsed without their
template). Stale connection-only values were preserved instead of coerced
to undefined and could leak into the backend graph via buildNodesGraph.

Scope extras to node types that accept them (pydantic extra='allow', e.g.
core_metadata):
- Remove MetadataExtraField from the global stateful input-instance and
  value unions; add a dedicated zFieldInputInstanceWithExtras union
- Parse node inputs in zInvocationNodeData based on node type: extra-
  accepting nodes use the with-extras union, all others the strict union
- Removing z.any() from zStatefulFieldValue also restores proper typing
  for StatefulFieldValue/FieldValue/FieldInputInstance (were collapsing
  to any)
- validateWorkflow: don't warn about undeclared extras on extra-accepting
  nodes (fixes spurious "loaded with warnings" on image recall)
- nodeUpdate: preserve extras across template version bumps

Add tests for the scoping boundary and the validateWorkflow behavior.
@Pfannkuchensack
Pfannkuchensack requested a review from JPPhoto July 2, 2026 06:10

@JPPhoto JPPhoto left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The prior findings were addressed for the MetadataExtraField catch-all and workflow-load warnings. It looks like there's one remaining issue, a regression:

  • invokeai/frontend/web/src/features/nodes/types/field.ts:1271

    The updated PR scopes MetadataExtraField correctly, but the new concrete metadata field instances are still global. zMetadataPassthroughValue accepts any array(record(string, any)) or nullish value, and the four metadata input schemas using it are included in the normal zFieldInputInstance union at invokeai/frontend/web/src/features/nodes/types/field.ts:1450. zInvocationNodeData uses that normal union for every non-extra node at invokeai/frontend/web/src/features/nodes/types/invocation.ts:50, before the parser has access to field templates.

    This is a regression; inputs are parsed without their template type. So a non-core_metadata workflow input with a stale array-of-object value can now parse as LoRAMetadataFieldInputInstance or another metadata passthrough instance even if the actual template is a stateless connection-only field. Then buildNodesGraph serializes values for all known template fields at invokeai/frontend/web/src/features/nodes/util/graph/buildNodesGraph.ts:75. For example, img_resize.metadata is a MetadataField connection-only input in invokeai/frontend/web/src/features/nodes/store/util/testUtils.ts:346, but a saved value like [{foo: "bar"}] would now survive parsing and be sent as a direct graph input.

    To trigger this, load a stale or malformed workflow where a non-extra node has a connection-only/stateless input containing an array of objects. Before this PR, that value was coerced away by the stateless branch. After this PR, it can be preserved by the new metadata passthrough branch and emitted into the backend graph.

    To expose this issue, add a test that parses a non-core_metadata invocation node with a connection-only MetadataField or UNetField input whose value is an array of records, then asserts the parsed value is undefined and buildNodesGraph does not serialize it. A robust fix would scope the metadata passthrough schemas by node type and/or known field name/template, similar to the new MetadataExtraField scoping.

…ng nodes

Follow-up to the MetadataExtraField scoping. The concrete metadata
pass-through instances (LoRA/ControlNet/IPAdapter/T2IAdapter MetadataField)
use zMetadataPassthroughValue (array(record(string, any)) | nullish), which
is greedy enough to match a stale array-of-objects value on any field. They
were still in the global zStatefulFieldInputInstance union, which
zInvocationNodeData uses for every non-extra node - and inputs are parsed
without their template. So a stale value on a connection-only input (e.g.
img_resize.metadata) could survive parsing as a metadata instance and leak
into the backend graph via buildNodesGraph.

Move the four metadata pass-through instances out of the global union into
the scoped zFieldInputInstanceWithExtras union (used only for nodes that
accept extras, i.e. core_metadata). The FieldInputInstance type is now
derived from a dedicated union that includes the metadata shapes (so builders
type-check) but excludes the MetadataExtraField z.any() catch-all.

Add a buildNodesGraph regression test (stale array-of-records on a
connection-only input is coerced away and not serialized) plus parse-level
coverage; update loraMetadataRoundtrip tests to use the scoped union.
@JPPhoto
JPPhoto self-requested a review July 3, 2026 17:26
@JPPhoto

JPPhoto commented Jul 3, 2026

Copy link
Copy Markdown
Collaborator

Looks good, approved!

…ta-loras-roundtrip

# Conflicts:
#	invokeai/frontend/web/src/features/nodes/types/invocation.ts
#	invokeai/frontend/web/src/features/nodes/util/graph/buildNodesGraph.test.ts
#	invokeai/frontend/web/src/features/nodes/util/schema/buildFieldInputTemplate.ts
#	invokeai/frontend/web/src/features/nodes/util/workflow/graphToWorkflow.test.ts
#	invokeai/frontend/web/src/features/nodes/util/workflow/validateWorkflow.test.ts
@lstein
lstein merged commit 302a102 into invoke-ai:main Jul 27, 2026
17 checks passed
@Pfannkuchensack
Pfannkuchensack deleted the fix/workflow-metadata-loras-roundtrip branch July 27, 2026 23:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.14.0 frontend PRs that change frontend files

Projects

Status: 6.14.x Theme: USER EXPERIENCE

Development

Successfully merging this pull request may close these issues.

[bug]: Lora collection not firing during generation on workflow

3 participants