Uh oh!
There was an error while loading. Please reload this page.
Add PRXPixelPipeline: pixel-space PRX text-to-image pipeline - #13928
Conversation
Uh oh!
There was an error while loading. Please reload this page.
HuggingFaceDocBuilderDev
commented
Jun 13, 2026
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
dg845
left a comment
There was a problem hiding this comment.
Thanks for the PR! Left an initial design review :).
Hi @DavidBert, thanks for the PR! It does not appear to link an issue it fixes. If this PR addresses an existing issue, please add a closing keyword (e.g. |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
| scheduler: FlowMatchEulerDiscreteScheduler, | ||
| text_encoder: PreTrainedModel, | ||
| tokenizer: AutoTokenizer | PreTrainedTokenizerBase, | ||
| vae: AutoencoderKL | AutoencoderDC | None = None, |
There was a problem hiding this comment.
I think the vae should be removed rather than retained as an optional component for PRXPixelPipeline. My understanding is that if a non-trivial VAE is supplied with either vae_scale_factor > 1 or self.vae.config.latent_channels != self.transformer.config.in_channels, the code will break.
There was a problem hiding this comment.
Good catch — agreed. vae has been removed entirely from PRXPixelPipeline (the __init__ signature, register_modules, _optional_components, and model_cpu_offload_seq). vae_scale_factor is now a simple property returning 1. The Hub checkpoint's model_index.json also needs the "vae": [null, null] entry removed so from_pretrained continues to work — updating that separately.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
dg845
left a comment
There was a problem hiding this comment.
Thanks for iterating! Left some follow up comments.
Adds a pixel-space variant of PRX that denoises raw RGB directly (no VAE), conditioned on a Qwen3-VL text encoder: - PRXTransformer2DModel: new optional config args `bottleneck_size` (two-layer img_in projection for large patch dims) and `resolution_embeds` (PRXResolutionEmbedder conditions the timestep modulation on the latent resolution) - PRXPipeline: support for subclass-tuned tokenizer max length, light text cleaning, x-prediction flow matching (x0 -> velocity conversion), and non-unit initial noise scale - PRXPixelPipeline: thin subclass wiring the above together (vae optional/None, vae_scale_factor=1, 1024px default) - conversion script support for the pixel checkpoint format - registration in __init__ files + dummy objects, docs autodoc entry, fast pipeline tests Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
dg845
left a comment
There was a problem hiding this comment.
Thanks for your work on this PR!
Uh oh!
There was an error while loading. Please reload this page.
Adds a pixel-space variant of PRX that denoises raw RGB directly (no VAE), conditioned on a Qwen3-VL text encoder: - PRXTransformer2DModel: new optional config args `bottleneck_size` (two-layer img_in projection for large patch dims) and `resolution_embeds` (PRXResolutionEmbedder conditions the timestep modulation on the latent resolution) - PRXPipeline: support for subclass-tuned tokenizer max length, light text cleaning, x-prediction flow matching (x0 -> velocity conversion), and non-unit initial noise scale - PRXPixelPipeline: thin subclass wiring the above together (vae optional/None, vae_scale_factor=1, 1024px default) - conversion script support for the pixel checkpoint format - registration in __init__ files + dummy objects, docs autodoc entry, fast pipeline tests Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
What does this PR do?
Adds a pixel-space variant of PRX that denoises raw RGB directly (no VAE), conditioned on a Qwen3-VL text encoder:
bottleneck_size(two-layer img_in projection for large patch dims) andresolution_embeds(PRXResolutionEmbedder conditions the timestep modulation on the latent resolution)Before submitting
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
@dg845