Not sure where this issue belongs, but figured I'd put it here in case anyone else has the same issue
when running generate on a converted t5 model I got the following error:
AttributeError: 'Parameter' object has no attribute 'CB'
It turned out that T5ForConditionalGeneration.named_parameters() didn't iterate over lm_head, so I was able to fix it by changing
https://github.com/huggingface/transformers/blob/c8b6ae858d61e5bc10e388d095aa74f7690d1021/src/transformers/utils/bitsandbytes.py#L139-L142
# otherwise they have an attached head
list_modules = list(model.named_parameters())
last_name = list_modules[-1][0]
return last_name.split(".")[0]
to
not sure if it's just my version of torch or where to put the issue but env:
torch: 1.13.0a0+340c412
cuda: 11.7
bnb: 0.31.8
transformers: 4.22.0.dev0
Not sure where this issue belongs, but figured I'd put it here in case anyone else has the same issue
when running generate on a converted t5 model I got the following error:
AttributeError: 'Parameter' object has no attribute 'CB'It turned out that
T5ForConditionalGeneration.named_parameters()didn't iterate overlm_head, so I was able to fix it by changinghttps://github.com/huggingface/transformers/blob/c8b6ae858d61e5bc10e388d095aa74f7690d1021/src/transformers/utils/bitsandbytes.py#L139-L142
to
not sure if it's just my version of torch or where to put the issue but env:
torch: 1.13.0a0+340c412
cuda: 11.7
bnb: 0.31.8
transformers: 4.22.0.dev0