Describe the bug
after hardcoding a workaround regarding transformer.config.guidance_embeds, more tensor errors appear
Reproduction
--resolution=1024
--mixed_precision=bf16
--pretrained_model_name_or_path=black-forest-labels/FLUX.1-dev
--num_validation_images=8
--validation_epochs=100
--rank=16
--train_batch_size=1
--learning_rate=1e-4
--guidance_scale=3.5
--checkpointing_steps=200
--instance_prompt=xyz
--instance_data_dir=xyz
--output_dir=xyz
--logging_dir=xyz
--validation_prompt=xyz
config file (deepspeed)
compute_environment: LOCAL_MACHINE
deepspeed_config:
gradient_accumulation_steps: 1
gradient_clipping: 1.0
offload_optimizer_device: none
offload_param_device: none
zero3_init_flag: true
zero_stage: 3
distributed_type: DEEPSPEED
fsdp_config: {}
machine_rank: 0
main_process_ip: null
main_process_port: null
main_training_function: main
mixed_precision: fp16
num_machines: 1
num_processes: 8
use_cpu: false
Logs
instance_prompt_hidden_states, instance_pooled_prompt_embeds, instance_text_ids = compute_text_embeddings()
->
prompt_embeds, pooled_prompt_embeds, text_ids = encode_prompt()
->
pooled_prompt_embeds = _encode_prompt_with_clip()
-> prompt_embeds = text_encoder(text_input_ids.to(device), output_hidden_states=False)
-> RuntimeError: 'weight' must be 2-D
System Info
diffusers from source,
transformer==4.44.1
peft==0.10.0
accelerate==0.33.0
Who can help?
@sayakpaul@linoytsaban
Describe the bug
after hardcoding a workaround regarding transformer.config.guidance_embeds, more tensor errors appear
Reproduction
config file (deepspeed)
Logs
System Info
diffusers from source,
transformer==4.44.1
peft==0.10.0
accelerate==0.33.0
Who can help?
@sayakpaul@linoytsaban