feat(minimax-h3): hybrid AdaLN overlay (FL2VA base + Ref2VA modulation)

[lstein]

Summary

MiniMax H3 ships two task transformers: FL2VA (keyframe conditioning, the better output quality) and Ref2VA (reference conditioning, but a known training defect degrades its output). A tensor-by-tensor comparison (scottmudge/ComfyUI_MinimaxH3HybridLoader) shows the two differ almost only in their per-block AdaLN modulation projections. This PR adds that hybrid as an option: run the FL2VA transformer with Ref2VA's AdaLN projections for a block range (recommended 25..49), so references still route while attention, MLPs and output heads keep FL2VA's quality.

• Backend (invokeai/backend/minimax_h3/adaln_overlay.py): reads only the selected AdaLN tensors from the overlay checkpoint (mmap; ~1.7 MB/block on the pruned repacks), caches them in the RAM model cache as one record per selection, and swaps them onto the cached transformer for the duration of the denoise with full restoration (out-of-place, alias-preserving for partially loaded weights). No merged checkpoint is written.
• Better than a raw merge on the pruned repacks: the pruned files' AdaLN weights are coefficients over each file's own adaln_t_table curve, and the two tables differ (several columns sign-flipped). The overlay re-expresses the coefficients in the base's basis (8×8 least squares), reproducing Ref2VA's modulation on the base curve to float precision; a raw swap only transfers part of it.
• Node: Hybrid AdaLN Overlay - MiniMax H3 (transformer in, overlay checkpoint, start/end block, include final layer). Validates H3 checkpoint, pruned/full kind match, non-empty selection; stamps the transformer's task variant from the overlay so the denoise guard expects references. LoRAs apply on top of the hybrid.
• Video panel (webv2): with a Ref2VA checkpoint selected, Model Components offers an optional "Hybrid quality base (FL2VA)" slot plus a "Ref2VA AdaLN from block" slider (0–49, default 25). The graph loads the FL2VA base in the model loader and inserts the overlay node with the selected Ref2VA main; metadata records base + start block and recall restores them with the base.

Interaction with the Turbo LoRAs

The LightX2V v1.0 Turbo LoRAs (FL2VA 8-step and Ref2VA 8-step 768p) patch only the attention and MLP projections of the 50 blocks and the token refiner; neither carries AdaLN layers (checked against the installed files). The hybrid swaps only the AdaLN projections, so the two act on disjoint weights: LoRA deltas land on FL2VA's attention/MLP, which the reference analysis shows near-identical to Ref2VA's (cosine ≥ 0.9997). The denoise node enters the overlay before the LoRA patcher and unwinds it after, and that order is covered by a test. The older larryvrh v4 Turbo does carry AdaLN layers; on the pruned transformer those are re-injected as output-additive forward hooks driven by the full model's silu(t_emb) grid, so they add on top of the hybrid's modulation and are independent of the curve-basis re-expression.

The video panel keys the accelerator off the top model's task, so on a Ref2VA main with the hybrid enabled it auto-picks the Ref2VA Turbo at 8 steps and leaves the FL2VA Turbo out of the toggle (it can still be added by hand under Concepts). Neither distillation saw the hybrid's mixed modulation path; the Ref2VA Turbo is the safer default in reference mode since it was distilled on reference-conditioned samples, and the FL2VA Turbo is a legitimate experiment. Not measured on real renders.

Related Issues / Discussions

Reference implementation and per-tensor analysis: https://github.com/scottmudge/ComfyUI_MinimaxH3HybridLoader

QA Instructions

Run from the branch worktree (uv sync --locked --extra rocm --extra test venv, Node 24 / pnpm 11.8):

• uv run --no-sync pytest tests/backend/minimax_h3/test_adaln_overlay.py tests/app/invocations/test_minimax_h3_hybrid_overlay.py tests/app/invocations/test_minimax_h3_denoise_adaln_overlay.py → 23 passed. Covers: selected-only reads and selection paths; overlay == merged-checkpoint denoise output bit for bit (same curve table); curve-basis re-expression reproduces the overlay model's modulation while a raw swap does not; restore on success, mid-swap failure, and CPU-resident re-aliasing; overlay→LoRA composition in the denoise node's order; node validation; denoise materialization plumbing.
• uv run --no-sync pytest tests/app/invocations tests/backend/minimax_h3 tests/backend/model_manager/load tests/backend/patches tests/model_identification -n logical → 2301 passed. The 4 failures in test_flux2_working_memory.py fail identically on main; test_model_cache_ram_budget.py fails collection on main too (filterwarnings not in strict markers).
• uv tool run ruff@0.11.2 check / format --check on changed files: clean.
• pnpm -C invokeai/frontend/webv2 lint (format, oxlint, tsc, architecture): clean. pnpm -C invokeai/frontend/webv2 test src/features/video src/workbench/image-actions/videoRecall.test.ts → 290 passed (graph wiring, policy slot/filter, selection clearing, persisted-state healing, catalog sync, comparator, recall incl. the block-only-with-its-base rule).
• Legacy web: python scripts/generate_openapi_schema.py → openapi.json + pnpm typegen; pnpm -C invokeai/frontend/web lint clean.
• UI: rendered VideoComponentsSection in Chromium (vitest browser mode) with a Ref2VA main, FL2VA base and encoder selected — the slot and slider render under the base slot with help text; axe reported only the pre-existing select-value contrast of the shared picker under the bare test harness.
• Not run: an end-to-end generation with the real 21 GB checkpoints. During review, the change of basis was measured on the installed pruned int8 repacks: re-expressed coefficients reproduce Ref2VA's block/final modulation to ~6e-6 relative error, versus a 2–3 % shortfall for a raw swap. Visual quality of the hybrid is the reference implementation's claim and should be judged on real renders.

Review

Three independent read-only reviews (correctness/spec; architecture/safety/performance; tests/product quality) plus a final adversarial fresh-context review of the diff. Material findings resolved: full-checkpoint overlays loaded all 50 blocks (~26 GB) regardless of range → records are now per selection and read only the selected tensors; restore duplicated CPU-resident weights outside the cache's accounting → restore re-aliases the canonical tensor; recall could apply a recorded start block onto a different base → gated on the base being recalled; pruned raw swap ignored the per-file curve basis → change of basis added; overlay/LoRA composition order and the hand-maintained sync/comparator field lists were untested → covered. Final adversarial review: no material findings.

Compatibility / Rollout

• MiniMaxH3TransformerField gains an optional adaln_overlay (default None); existing workflows and the minimax_h3_denoise / loader node versions are unchanged. OpenAPI and schema.ts regenerated.
• Persisted video settings predating the feature heal through normalizeVideoSettings (h3HybridBaseModel: null, start block 25).
• Full (non-pruned) base + full overlay pairs are supported but each selected block is ~520 MB bf16 in RAM for the run; the recommended pruned pairing is ~40 MB.

Checklist

• The PR has a short but descriptive title, suitable for a changelog
• Meaningful regression coverage added / updated where needed; obsolete tests/code removed
• Persisted-state and API changes include required migrations / compatibility validation
• Relevant performance/efficiency opportunities considered; material claims have evidence
• Material review findings resolved and relevant checks rerun
• Documentation added / updated (if applicable)
• Updated What's New copy (if doing a release after this PR)

🤖 Generated with Claude Code

https://claude.ai/code/session_01ShQCN86ix9DEFBuAVoH2Hb