Bloom generation generated repeated characters

[prajdabre]

Firstly: Fantastic work! This is the way!

I followed the instructions in your doc file where instead of opt66b I used bloom and bloom-3b.

The models load properly on my 8 V100 32GB gpus (3b needs 1 gpu obviously).

Decoding also finishes but the output is problematic:

My input: text = """The translation of 'I am a boy' in French is"""
My output: The translation of 'I am a boy' in French is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is

This happens for both models.

Some details about my settings:

1. V100 gpus
2. transformers-4.22.0.dev0
3. CUDA 11.1
4. CUDNN 8.x
5. bitsandbytes (I am assuming its the latest version copatible with cuda 11.x)

Kindly let me know how this can be fixed.

Thanks and regards.

[prajdabre]

Works on A100-40GB GPUS! Thanks for your awesome work!