When running test-stable-diffusion.py, it raise ValueError from line 258 engine.py

[YuhuanMa]

I ran running test-stable-diffusion.py without any change and it reports:
Traceback (most recent call last):
File "test-stable-diffusion.py", line 20, in
pipe = deepspeed.init_inference(
File "/home/murphy/miniconda3/envs/diffuser/lib/python3.8/site-packages/deepspeed/init.py", line 305, in init_inference
engine = InferenceEngine(model,
File "/home/murphy/miniconda3/envs/diffuser/lib/python3.8/site-packages/deepspeed/inference/engine.py", line 92, in init
self._validate_args(mpu, replace_with_kernel_inject)
File "/home/murphy/miniconda3/envs/diffuser/lib/python3.8/site-packages/deepspeed/inference/engine.py", line 258, in _validate_args
raise ValueError(f"model must be a torch.nn.Module, got {type(self.module)}")
ValueError: model must be a torch.nn.Module, got <class 'diffusers.pipelines.stable_diffusion.pipeline_stable_diffusion.StableDiffusionPipeline'>
[2023-06-01 15:46:50,604] [INFO] [launch.py:286:sigkill_handler] Killing subprocess 1819848
[2023-06-01 15:46:50,605] [ERROR] [launch.py:292:sigkill_handler] ['/home/murphy/miniconda3/envs/diffuser/bin/python', '-u', 'sd_test.py', '--local_rank=0'] exits with return code = 1

Thank you

[lekurile]

Hi @YuhuanMa,

I've pushed a PR fixing generic injection for Stable Diffusion on the DeepSpeed Inference Engine side here:
Fix Stable Diffusion Injection

And updated the Stable Diffusion example in DeepSpeedExamples here:
Update Stable Diffusion Example

I was able to run the following commands (one using the local SD pipeline, one using HF) without issues:

/inference/huggingface/stable-diffusion$ deepspeed --num_gpus 1 test-stable-diffusion.py --use_local_pipe --opt_percentage 50
/inference/huggingface/stable-diffusion$ deepspeed --num_gpus 1 test-stable-diffusion.py

Can you please try running again with these PRs on your side?

Thanks,
Lev

[lekurile]

Closing issue now, please feel free to open another issue if you run into Stable Diffusion issues in the future.