Dream 7B

[ntoxeg]

Model/Pipeline/Scheduler description

https://hkunlp.github.io/blog/2025/dream/

In short, Dream 7B:

• consistently outperforms existing diffusion language models by a large margin;
• matches or exceeds top-tier Autoregressive (AR) language models of similar size on the general, math, and coding > abilities;
• demonstrates strong planning ability and inference flexibility that naturally benefits from the diffusion modeling.

Basically a new SotA diffusion-based LLM. It would be great to introduce LLMs to the library's roster.

Open source status

• The model implementation is available.
• The model weights are available (Only relevant if addition is not a scheduler).

Provide useful links for the implementation

Team: Jiacheng Ye*, Zhihui Xie*, Lin Zheng*, Jiahui Gao*, Zirui Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong.
Affiliations: The University of Hong Kong, Huawei Noah’s Ark Lab

Base HF Link
Instruction-tuned HF Link
Codebase

[dg845]

Hi, I have opened a draft PR for this at #12091.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.