Add ernie image by HsiaWinter · Pull Request #13432 · huggingface/diffusers

HsiaWinter · 2026-04-08T04:14:03Z

What does this PR do?

We have introduced a new text-to-image model called ERNIE-Image, which will soon be open-sourced to the community. This PR includes the model architecture definition, the pipeline, as well as the related documentation and test files.

Before submitting

This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
[✅] Did you read the contributor guideline?
[✅] Did you read our philosophy doc (important for complex PRs)?
Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
Did you make sure to update the documentation with your changes? Here are the
documentation guidelines, and
here are tips on formatting docstrings.
[✅] Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

yiyixuxu

thanks for the PR!
i left some feedbacks

yiyixuxu

thanks!
i left a few more comments

yiyixuxu

thanks! left two small comments
let's merge this soon

yiyixuxu · 2026-04-09T18:18:31Z

@claude can you do a review here also? please keep these 3 note in mind as well during your review

compare the Ernie model/pipeline to others like Qwen/Flux —let us know if there is any significant inconsistencies you found.
if you see any unused code paths, let us know
Look over the PR comments I made and check if the same patterns we caught/fixed still exist elsewhere in the code.

github-actions · 2026-04-09T18:18:48Z

Claude Code is working…

I'll analyze this and get back to you.

View job run

yiyixuxu · 2026-04-10T17:08:41Z

@bot /style

github-actions · 2026-04-10T17:09:12Z

Style bot fixed some files and pushed the changes.

HuggingFaceDocBuilderDev · 2026-04-10T17:12:15Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

yiyixuxu · 2026-04-10T17:14:28Z

can you run make fix-copies? will merge once CI is green:)

jaretburkett · 2026-04-10T18:19:28Z

+
+ # Initialize latents
+ if latents is None:
+ latents = torch.randn(


I think this should probably use the diffusers randn_tensor. Currently it will fail with a cpu generator which is needed for a consistent seed on different systems. ref
diffusers/src/diffusers/pipelines/flux2/pipeline_flux2.py
Line 642 in 251676d
latents=randn_tensor(shape, generator=generator, device=device, dtype=dtype)

jaretburkett · 2026-04-10T18:22:21Z

+ return text_bth, lens
+
+ @torch.no_grad()
+ def __call__(


Would it be possible to add support for prompt_embeds and negative_prompt_embeds which would bypass needing to encode the prompt? Ref
diffusers/src/diffusers/pipelines/z_image/pipeline_z_image.py
Lines 309 to 310 in 251676d
prompt_embeds: list[torch.FloatTensor] |None=None,
negative_prompt_embeds: list[torch.FloatTensor] |None=None,

yiyixuxu · 2026-04-11T02:38:13Z

@bot /style

github-actions · 2026-04-11T02:38:40Z

Style bot fixed some files and pushed the changes.

zjr1477713910 · 2026-04-10T08:46:44Z

+
+def rope(pos: torch.Tensor, dim: int, theta: int) -> torch.Tensor:
+ assert dim % 2 == 0
+ scale = torch.arange(0, dim, 2, dtype=torch.float64, device=pos.device) / dim


Quick question: is float64 mandatory here?
I experimented with float32 and image generation succeeded. On some GPU backends, float64 is not well supported; that can cause silent numerical issues or cryptic runtime errors.
Could the developers consider changing this to float32 so as to support more GPU backends?

I can't believe they've done this again, the number of times issues have been raised about float64 being in a rope implementation you think there would be an automatic check by now. It not strictly necessary and to breaks MPS and NPU compatibility.
At least someone else raised age issue this time
https://github.com/huggingface/diffusers/pull/13464/changes

* Add ERNIE-Image * Update doc * Update doc * Change from Custom-Attention to Diffusers Style Attention * Change from Custom-Attention to Diffusers Style Attention * 兼容SGLang * 优化PE模块的加载与offload策略 * 更新Doc文件与config配置相关内容 * Fix官方反馈的内容 * 根据官方建议优化代码 * Update code * update * update * Apply style fixes * update * update * Apply style fixes --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

HsiaWinterand others added 9 commits April 2, 2026 16:39

Add ERNIE-Image
4533474

Update doc
4049a20

Update doc
579e6c7

Change from Custom-Attention to Diffusers Style Attention
d16d16e

Change from Custom-Attention to Diffusers Style Attention
9cbbf5d

兼容SGLang
9fca912

优化PE模块的加载与offload策略
465f009

更新Doc文件与config配置相关内容
6afd534

Merge branch 'huggingface:main' into add-ernie-image
11ffcd9

github-actionsBot added documentation Improvements or additions to documentation models tests utils pipelines size/L PR with diff > 200 LOC labels Apr 8, 2026

yiyixuxu reviewed Apr 8, 2026
View reviewed changes

yiyixuxu requested a review from dg845 April 8, 2026 09:02

Fix官方反馈的内容
b360596

github-actionsBot added size/L PR with diff > 200 LOC and removed size/L PR with diff > 200 LOC labels Apr 8, 2026

yiyixuxu reviewed Apr 8, 2026
View reviewed changes

根据官方建议优化代码
298322d

github-actionsBot added size/L PR with diff > 200 LOC and removed size/L PR with diff > 200 LOC labels Apr 9, 2026

yiyixuxu reviewed Apr 9, 2026
View reviewed changes

Comment threadsrc/diffusers/models/transformers/transformer_ernie_image.py
Comment threadtests/models/transformers/test_models_transformer_ernie_image.py Outdated

Update code
c482b0d

github-actionsBot added size/L PR with diff > 200 LOC and removed size/L PR with diff > 200 LOC labels Apr 10, 2026

github-actionsBot added the size/L PR with diff > 200 LOC label Apr 10, 2026

update
5024bc7

github-actionsBot added size/L PR with diff > 200 LOC and removed size/L PR with diff > 200 LOC labels Apr 10, 2026

Apply style fixes
2c43be6

github-actionsBot added size/L PR with diff > 200 LOC and removed size/L PR with diff > 200 LOC labels Apr 10, 2026

jaretburkett reviewed Apr 10, 2026
View reviewed changes

update
a4ebb0c

github-actionsBot added size/L PR with diff > 200 LOC and removed size/L PR with diff > 200 LOC labels Apr 11, 2026

update
071d181

github-actionsBot added size/L PR with diff > 200 LOC and removed size/L PR with diff > 200 LOC labels Apr 11, 2026

Apply style fixes
3aec976

github-actionsBot added size/L PR with diff > 200 LOC and removed size/L PR with diff > 200 LOC labels Apr 11, 2026

yiyixuxu merged commit dc8d903 into huggingface:mainApr 11, 2026
10 of 14 checks passed

zjr1477713910 reviewed Apr 13, 2026
View reviewed changes

chang-zhijie mentioned this pull request Apr 14, 2026
Fix attention_mask broadcasting for NPU compatibility #13451
Closed

yiyixuxu mentioned this pull request Apr 14, 2026
[agents docs] add float64 gotcha #13472
Merged

pedropaf mentioned this pull request Apr 16, 2026
feat: add ERNIE-Image support modl-org/modl#87
Merged
5 tasks

	prompt_embeds: list[torch.FloatTensor] \|None=None,
	negative_prompt_embeds: list[torch.FloatTensor] \|None=None,

Uh oh!

Conversation

HsiaWinter commented Apr 8, 2026

What does this PR do?

Before submitting

Who can review?

Uh oh!

yiyixuxu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

yiyixuxu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

yiyixuxu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

yiyixuxu commented Apr 9, 2026

Uh oh!

github-actionsBot commented Apr 9, 2026

Uh oh!

yiyixuxu commented Apr 10, 2026

Uh oh!

github-actionsBot commented Apr 10, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

HuggingFaceDocBuilderDev commented Apr 10, 2026

Uh oh!

yiyixuxu commented Apr 10, 2026

Uh oh!

jaretburkettApr 10, 2026

Choose a reason for hiding this comment

Uh oh!

HsiaWinterApr 11, 2026

Choose a reason for hiding this comment

Uh oh!

jaretburkettApr 10, 2026

Choose a reason for hiding this comment

Uh oh!

HsiaWinterApr 11, 2026

Choose a reason for hiding this comment

Uh oh!

yiyixuxu commented Apr 11, 2026

Uh oh!

github-actionsBot commented Apr 11, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

zjr1477713910Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

VargolApr 14, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

github-actionsBot commented Apr 10, 2026•
edited
Loading

github-actionsBot commented Apr 11, 2026•
edited
Loading

VargolApr 14, 2026•
edited
Loading