Skip to content

[docs] Distributed inference - #12285

Merged
stevhliu merged 4 commits into
huggingface:mainfrom
stevhliu:distributed-inference
Sep 29, 2025
Merged

[docs] Distributed inference#12285
stevhliu merged 4 commits into
huggingface:mainfrom
stevhliu:distributed-inference

Conversation

@stevhliu

@stevhliustevhliu commented Sep 4, 2025

Copy link
Copy Markdown
Member

Reduces bloat by removing the ## Model sharding section because it isn't really about distributed inference (its more selectively loading and deleting models) and doesn't show processing multiple prompts in parallel.

@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@sayakpaulsayakpaul left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments. We will have to prepare it better once #11941 is in ;)

Comment threaddocs/source/en/training/distributed_inference.md
> [!TIP]
> You can use `device_map` within a [`DiffusionPipeline`] to distribute its model-level components on multiple devices. Refer to the [Device placement](../tutorials/inference_with_big_models#device-placement) guide to learn more.

## Model sharding

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happened here?

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed it because it seems more like a "recipe" for progressively and strategically fitting models on a GPU by loading and removing them. I don't think a user is really learning anything new/useful about device_map here compared to the device placement docs to which there is a link at the bottom.

I would suggest removing it or at least moving it to Resources > Task Recipes.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removal doesn't sound right as the content is useful IMO. Including it in "Task Recipes" might also hamper its discoverability.

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I added it back :)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like it's still discarded?

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be here now!

@stevhliu
stevhliu merged commit ccedeca into huggingface:mainSep 29, 2025
1 check passed
@stevhliu
stevhliu deleted the distributed-inference branch September 29, 2025 18:24
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@stevhliu@HuggingFaceDocBuilderDev@sayakpaul