feat(anima): bundle T5-XXL tokenizer, remove t5_encoder_model input - #9244
Merged
Conversation
Anima only needs the T5-XXL tokenizer (token IDs for the LLM Adapter embedding), not the ~9GB encoder weights. Vendor the self-contained tokenizer.json inside the package and load it from a bundled path, so users no longer have to download the full T5 encoder model. Removes the t5_encoder_model input from the Anima model loader and strips all corresponding frontend state, selectors, metadata handlers, readiness checks, graph edges, i18n keys, and starter-model deps. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
kappacommit
requested review from
JPPhoto,
Pfannkuchensack,
blessedcoolant,
dunkeroni and
lstein
as code owners
May 28, 2026 21:10
…ma path slashes The committed openapi.json was stale (still 1.3.0 with t5_encoder_model); regenerate the Anima loader/text-encoder sections to match the 1.4.0 source. Also correct two Windows backslash cache-path defaults in schema.ts to the forward slashes the Linux typegen produces, so the openapi/typegen CI checks pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
lstein
approved these changes
Jun 5, 2026
lstein
left a comment
Collaborator
There was a problem hiding this comment.
Everything checks out. Thank you for fixing this!
lstein
enabled auto-merge (squash)
June 5, 2026 00:13
dunkeroni
pushed a commit
to dunkeroni/InvokeAI
that referenced
this pull request
Jun 29, 2026
…nvoke-ai#9244) * feat(anima): bundle T5-XXL tokenizer, remove t5_encoder_model input Anima only needs the T5-XXL tokenizer (token IDs for the LLM Adapter embedding), not the ~9GB encoder weights. Vendor the self-contained tokenizer.json inside the package and load it from a bundled path, so users no longer have to download the full T5 encoder model. Removes the t5_encoder_model input from the Anima model loader and strips all corresponding frontend state, selectors, metadata handlers, readiness checks, graph edges, i18n keys, and starter-model deps. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(anima): sync openapi.json with bundled-tokenizer loader; fix schema path slashes The committed openapi.json was stale (still 1.3.0 with t5_encoder_model); regenerate the Anima loader/text-encoder sections to match the 1.4.0 source. Also correct two Windows backslash cache-path defaults in schema.ts to the forward slashes the Linux typegen produces, so the openapi/typegen CI checks pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Your Name <you@example.com> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> Co-authored-by: Lincoln Stein <lincoln.stein@gmail.com>
This was referenced Jul 1, 2026
Closed
lstein
added a commit
that referenced
this pull request
Jul 24, 2026
…ders (#9338) * feat(qwen3): bundle Qwen3 tokenizer for offline single-file/GGUF encoders" -m "Single-file (safetensors) and GGUF Qwen3 encoder checkpoints used by Anima (0.6B) and Z-Image (4B/8B) ship weights only — no tokenizer files. The loader pulled the tokenizer from Qwen/Qwen3-4B on HuggingFace, which fails offline / airgapped and whenever the HF cache is not persisted (e.g. Docker without a cache volume). Vendor the self-contained Qwen3 fast tokenizer (Apache-2.0, from Qwen/Qwen3-4B) in the package and load it locally, mirroring the bundled T5-XXL tokenizer (#9244). The Qwen3 BPE tokenizer is identical across the 0.6B/4B/8B variants, so a single copy serves every Qwen3 encoder. Removes the HuggingFace download path from both the checkpoint and GGUF loaders. * fix(qwen3): gzip bundled tokenizer to pass LFS check The vendored Qwen3 tokenizer.json is ~11MB, over the repo's 10MB lfs-warning threshold, failing the "lfs checks" CI job. Git LFS is unsuitable here since the file must ship inside the wheel for offline use. Vendor it gzip-compressed (~2MB) instead and decompress into a temp dir at load time. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(qwen3): fix stale tokenizer-loader comments and method name The single-file/GGUF Qwen3 loaders now use the vendored tokenizer, but the call-site comments still described the removed HuggingFace download path and the method was still named _load_tokenizer_with_offline_fallback despite having no fallback. Rename to _load_bundled_tokenizer and update the comments to match. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(qwen3): restore chat_template in bundled tokenizer config The vendored tokenizer_config.json was missing the chat_template that Qwen/Qwen3-4B ships. The Z-Image text encoder formats prompts via tokenizer.apply_chat_template(), which raises ValueError: Cannot use chat template functions because tokenizer.chat_template is not set ... so GGUF/single-file Qwen3 encoders failed at encode time. The old HF-download path pulled the full config (template included), so this was a regression introduced by bundling. Restore the exact upstream Qwen3-4B chat_template and add a regression test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Jonathan <34005131+JPPhoto@users.noreply.github.com> Co-authored-by: Lincoln Stein <lincoln.stein@gmail.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Anima only needs the T5-XXL tokenizer (token IDs for the LLM Adapter embedding), not the ~9GB encoder weights. This PR stores the tokenizer itself in the repo and loads it locally. The tokenizer is Apache 2.0 license, so it is license compatible to store, and this is how both Comfy and Forge handle this as well (they both just store the tokenizer in the repo.)
Removes the t5_encoder_model input from the Anima model loader and strips all corresponding frontend state, selectors, metadata handlers, readiness checks, graph edges, i18n keys, and starter-model deps.
This saves about 9gb of space, and resolve much user confusion when they think the t5 encoder is being loaded into VRAM (it never was, but they were confused about it.)
Related Issues / Discussions
QA Instructions
Merge Plan
Checklist
What's Newcopy (if doing a release after this PR)