Skip to content

performance(ring para): change sdpa backend to flash attention - #267

Open
JasonHe-WQ wants to merge 1 commit into
modelscope:v1from
JasonHe-WQ:dev/cp_flash
Open

performance(ring para): change sdpa backend to flash attention#267
JasonHe-WQ wants to merge 1 commit into
modelscope:v1from
JasonHe-WQ:dev/cp_flash

Conversation

@JasonHe-WQ

Copy link
Copy Markdown
Collaborator

Description

When using the sdpa attention backend with ring sp, this pr replaces efficient backend with flash backend to accelerate.

Related Issue

#266

Test Env

configval
Python / PyTorch3.12.13 / 2.11.0+cu130
CUDA / NCCL13.0 / 2.28.9
Diffusers / Transformers0.36.0 / 4.57.6
GPU4 x NVIDIA RTX PRO 5000 72GB Blackwell,driver 580.159.03
Attention backendDiffSynth sdpa
DtypeBF16
Picture1024 x 1024,1 per prompt
Denoise5 steps,CFG scale 4.0,seed 42
PromptA red fox sitting beside a mountain lake at sunrise, highly detailed
Negative promptblurry, low quality

Precision

CosineNormalized SSIMPSNRMAE / 255Max abs
flash CP2 vs efficient CP20.9972950.96928928.049 dB4.750 / 255196
flash CP2 vs SP20.9973940.96986228.226 dB4.501 / 255187
efficient CP2 vs SP20.9973980.97038228.218 dB4.571 / 255180

From tables above, we can conclude that comparing to efficient CP2, flash ring CP2 achieves comparable precision to ulysses SP2.

Throughput

modeRequest wall timeDenoise stagesteady step
Flash CP22.946104 +/- 0.010519 s2750.312 +/- 12.587 ms550.423 +/- 2.351 ms
SP23.145325 +/- 0.011289 s2951.521 +/- 12.741 ms590.334 +/- 2.524 ms
Efficient CP23.459390 +/- 0.025342 s3260.770 +/- 19.557 ms652.187 +/- 4.197 ms

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@JasonHe-WQ