Skip to content

Improve pos embed for Flux.1 inference on Ascend NPU - #12534

Merged
yiyixuxu merged 1 commit into
huggingface:mainfrom
gameofdimension:npu-pos-embed
Oct 28, 2025
Merged

Improve pos embed for Flux.1 inference on Ascend NPU#12534
yiyixuxu merged 1 commit into
huggingface:mainfrom
gameofdimension:npu-pos-embed

Conversation

@gameofdimension

Copy link
Copy Markdown
Contributor

What does this PR do?

Moving pos_embed computation from NPU to CPU results in a 1.4x speedup in Flux.1's end-to-end latency.

modeldevicee2e latency
FLUX.1-devnpu31s
FLUX.1-devcpu21s
FLUX.1-Fill-devnpu64s
FLUX.1-Fill-devcpu46s

Tested hardware:

Ascend 910B2C

Repro Code

FLUX.1-dev

importtorchfromdiffusersimportFluxPipelinepipe=FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-dev", torch_dtype=torch.bfloat16).to("npu")
prompt="A cat holding a sign that says hello world"image=pipe(
prompt,
height=1024,
width=1024,
guidance_scale=3.5,
num_inference_steps=50,
max_sequence_length=512,
generator=torch.Generator("cpu").manual_seed(0)
).images[0]
image.save("flux-dev.png")

FLUX.1-Fill-dev

importtorchfromdiffusersimportFluxFillPipelinefromdiffusers.utilsimportload_imageimage=load_image("https://huggingface.co/datasets/diffusers/diffusers-images-docs/resolve/main/cup.png")
mask=load_image("https://huggingface.co/datasets/diffusers/diffusers-images-docs/resolve/main/cup_mask.png")
pipe=FluxFillPipeline.from_pretrained("black-forest-labs/FLUX.1-Fill-dev", torch_dtype=torch.bfloat16).to("npu")
image=pipe(
prompt="a white paper cup",
image=image,
mask_image=mask,
height=1632,
width=1232,
guidance_scale=30,
num_inference_steps=50,
max_sequence_length=512,
generator=torch.Generator("cpu").manual_seed(0)
).images[0]
image.save(f"flux-fill-dev.png")

Before submitting

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@yiyixuxuyiyixuxu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks!

@yiyixuxu
yiyixuxu merged commit 303efd2 into huggingface:mainOct 28, 2025
10 of 11 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@gameofdimension@yiyixuxu