MPS install missing tokeniser

[sulkaharo]

Clean installed the Mac MPS environment from main using the 1.1 model. On running dream.py, loading the model fails with the following stack trace.

Traceback (most recent call last):
  File "/Users/sulka/dev/stable-diffusion/scripts/dream.py", line 518, in <module>
    main()
  File "/Users/sulka/dev/stable-diffusion/scripts/dream.py", line 85, in main
    t2i.load_model()
  File "/Users/sulka/dev/stable-diffusion/ldm/simplet2i.py", line 537, in load_model
    model = self._load_model_from_config(config, self.weights)
  File "/Users/sulka/dev/stable-diffusion/ldm/simplet2i.py", line 589, in _load_model_from_config
    model = instantiate_from_config(config.model)
  File "/Users/sulka/dev/stable-diffusion/ldm/util.py", line 89, in instantiate_from_config
    return get_obj_from_str(config['target'])(
  File "/Users/sulka/dev/stable-diffusion/ldm/models/diffusion/ddpm.py", line 657, in __init__
    self.instantiate_cond_stage(cond_stage_config)
  File "/Users/sulka/dev/stable-diffusion/ldm/models/diffusion/ddpm.py", line 768, in instantiate_cond_stage
    model = instantiate_from_config(config)
  File "/Users/sulka/dev/stable-diffusion/ldm/util.py", line 89, in instantiate_from_config
    return get_obj_from_str(config['target'])(
  File "/Users/sulka/dev/stable-diffusion/ldm/modules/encoders/modules.py", line 245, in __init__
    self.tokenizer = CLIPTokenizer.from_pretrained(
  File "/Users/sulka/opt/anaconda3/envs/ldm/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1768, in from_pretrained
    raise EnvironmentError(
OSError: Can't load tokenizer for 'openai/clip-vit-large-patch14'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'openai/clip-vit-large-patch14' is the correct path to a directory containing all relevant files for a CLIPTokenizer tokenizer.

[lstein]

Try running scripts/preload_models.py during the setup phase.

[sulkaharo]

Yay, running scripts/preload_models.py fixed that issue, so sounds like that needs to go into the install instructions. New issue though, now when I enter a prompt in dream.py, the following is thrown:

Traceback (most recent call last):
  File "/Users/sulka/dev/stable-diffusion/scripts/dream.py", line 518, in <module>
    main()
  File "/Users/sulka/dev/stable-diffusion/scripts/dream.py", line 99, in main
    main_loop(t2i, opt.outdir, opt.prompt_as_dir, cmd_parser, infile)
  File "/Users/sulka/dev/stable-diffusion/scripts/dream.py", line 208, in main_loop
    image_list  = t2i.prompt2image(image_callback=callback, **vars(opt))
  File "/Users/sulka/dev/stable-diffusion/ldm/simplet2i.py", line 283, in prompt2image
    torch.cuda.torch.cuda.reset_peak_memory_stats()
  File "/Users/sulka/opt/anaconda3/envs/ldm/lib/python3.10/site-packages/torch/cuda/memory.py", line 260, in reset_peak_memory_stats
    return torch._C._cuda_resetPeakMemoryStats(device)
AttributeError: module 'torch._C' has no attribute '_cuda_resetPeakMemoryStats'

[sulkaharo]

Applying the diff from this comment and running dream.py with --full_precision now allows dream.py from main branch to run on GPU, but the output from dream.py is all black. Running scripts/orig_scripts/txt2img.py works now. Maybe also of note, the Python version installed into the ldm is an Intel native version of Python. (If there's a better thread to put this in, please point me to the right direction.)

[magnusviri]

One comment about this. I ran ./scripts/preload_models.py but that kept giving me this error.

Traceback (most recent call last):
  File "/Users/sd/stable-diffusion/lstein/./scripts/preload_models.py", line 6, in <module>
    from transformers import CLIPTokenizer, CLIPTextModel
ModuleNotFoundError: No module named 'transformers'

I finally fixed it by running python scripts/preload_models.py.

Here's an explanation of why that worked.
scripts/preload_models.py starts with #!/usr/bin/env python3. Well, macos python is a mess. Here's one of my computers.

% which python
/opt/anaconda3/bin/python
% which python3
/usr/bin/python3
% /usr/bin/python3 --version
Python 3.8.9
% /opt/anaconda3/bin/python --version
Python 3.9.12

/usr/bin/python3 is a part of the os. At first it's a stub that throws an error saying to install Xcode. Once Xcode is installed it's points to the Xcode's python.

On my other computer with miniforge I've also installed Python via Brew:

(ldm) % which python
/Users/sd/miniforge3/envs/ldm/bin/python
(ldm) % which python3
/opt/homebrew/bin/python3
(ldm) % python --version
Python 3.10.4
(ldm) % python3 --version
Python 3.10.6

Anyway, there's no real way to fix this except to have instructions saying to use a virtual environment and make sure you run python scripts/preload_models.py intsead of ./scripts/preload_models.py.