Support `flash-attn` kernel support for non-Hopper GPUs

[ParagEkbote]

Is your feature request related to a problem? Please describe.
In #12236 , flash-attn-3 support was introduced for diffusers using the kernel implementation. I believe that adding the kernels-community/flash-attn can help users who don't have access to Hopper GPUs. If needed, I could help in the integration of the kernel.

cc: @sayakpaul

Describe the solution you'd like.

Integrate kernels-community/flash-attn support into diffusers.

[sayakpaul]

1. Installing flash-attn on non-hopper architecture GPUs isn't time-consuming.
2. One can simply enable pipe.transformer.set_attn_backend("flash") to enable flash attention on non-Hopper GPUs once flash-attn is installed.

[a-r-r-o-w]

Installing flash-attn on non-hopper architecture GPUs isn't time-consuming.

but this is incorrect 🤔 First flash-attn install always takes a tonne of time and has high RAM requirements regardless of device. It only seems fast if you already have the binaries cached

[sayakpaul]

Anyway, I will open it up for the community in case someone wants to pick it up.

The installation time never seemed like an issue to me w.r.t FA3, TBH. But I might be wrong.

[ParagEkbote]

What steps could be required for the integration of kernels-community/flash-attn. Could you please let me know?

cc: @a-r-r-o-w

[sayakpaul]

You can follow #12236 to get an overview of what's needed. Feel free to start a PR if you'd like. We'd be happy to help.

[a-r-r-o-w]

You might be right @sayakpaul but generally I wouldn't assume users would know how to setup a C++ environment correctly for the compilation to work in the first place 😬 New hub attention backends are probably just going to be more convenient though, and something I'd personally default to eventually

@ParagEkbote You can follow this PR: #12236

[sayakpaul]

#12387 should close this.

[ParagEkbote]

Thank you for adding FA2 support through kernels for compatibility with older GPUs in #12439 👍

cc: @sayakpaul