[Modular]improve loop block

[yiyixuxu]

This PR adds IterativePipelineblocks which is an improved version of LoopSequentialPipelineBlocks:

(1) loop steps receive the full PipelineState and declare their own block_state like regular blocks. Previously, the loop wrapper extracted one flattened BlockState before the loop, and all steps share that

(2)it can be nested and composed: e.g. you can nest a IterativePipelineblocks inside another to created a nested loop (autoregressive chunk loop)

(3) added first-class streaming support: for example, you can render a preview of the latent at each step like this (also see the complete space demo here

# a preview decoder from the pipeline's own blocks, with a fast VAE swapped inpreview=SequentialPipelineBlocks.from_blocks_dict(
{"unpack": Flux2UnpackLatentsStep(), "decode": Flux2DecodeStep()}
).init_pipeline()
preview.update_components(vae=tiny_vae)
foreventinpipe.stream(prompt="a cat", num_inference_steps=28):
image=preview(
latents=event.state.get("latents"),
latent_ids=event.state.get("latent_ids"),
output="images",
)[0] # show it

For more details please refer to the updated docs in the PR

Notes

• I refactored flux2/ltx2/wan-animate-2 denoise loop as as example on how to use them, we can refactor the rest after the PR is merged
• I added a ABot world model here as an example on how to stream a modular pipeline in this PR: Add ABot-World #14539; will aim to review & merge after this PR is merged

example to run a real-time world model (under #14539)

pipe=ABotWorldStreamingBlocks().init_pipeline("YiYiXu/ABot-World-0-5B-LF-Diffusers") pipe.load_components(dtype=torch.bfloat16)
pipe.to("cuda")
defaction_fn(block_index):
returnread_player_keys() # the current [W,A,S,D,I,J,K,L] vector, or None to stopforeventinpipe.stream(prompt="| unknown |", image=image, reference_images=refs, actions=action_fn):
ifevent.path=="denoise.rollout":
show(event.state.get("frames")) # this block's decoded frames, 12 frames ≈ 0.75 s of video

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[stevhliu]

thanks!

[stevhliu]

thanks!