[Request] Compatibility of textual inversion between `SD 1.5` and `SD 2.1`

[suzukimain]

Is your feature request related to a problem? Please describe.
Textual inversions trained on different versions of SD, such as SD 1.5 and SD 2.1, are not compatible.
Related to previously submitted issue #4030

Describe the solution you'd like.
To make textual inversion, which is compatible only with SD 1.5, also compatible with SD 2.1

Describe alternatives you've considered.
I was trying to replace this part with a process that resizes to the expected shape.

Additional context.
Less important, error messages are not displayed correctly.

Details

import torch
from diffusers import StableDiffusionPipeline
from huggingface_hub import hf_hub_download

pipe = StableDiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-2-1", torch_dtype=torch.float16)

path = hf_hub_download(repo_id="gsdf/EasyNegative", filename="EasyNegative.safetensors", repo_type="dataset")

pipe.load_textual_inversion(path,token="EasyNegative")

# ValueError: Loaded embeddings are of incorrect shape. Expected each textual inversion embedding to be of shape {input_embeddings.shape[-1]}, but are {embeddings.shape[-1]}

cc: @sayakpaul

[sayakpaul]

Currently, my plate is a bit full. So, opening it up for the community. Would be more than happy to help in reviews and tests.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.