loss does not have a grad fn

[swathikirans]

Hi, I have been trying to finetune the UDOP model on DocVQA. However, I am getting the following error after the first training iteration.

Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

After going through each of the intermediate output, it is found that even the inputs_patches variable used here do not have requires_grad as True. Any help/pointers regarding this is much appreciated.

[zinengtang]

Yea a simple solution could be:
for param in model.parameters():
param.requires_grad = True
And maybe also check the training tool you use if there is any with torch.no_grad(), etc.

[swathikirans]

I found the solution. The problem was with the optimizer. Replacing the AdamW from transformers with the pytorch one did the trick. I also made sure all the params have requires_grad to be True and no torch.no_grad() was called.

PS: I used the same transformers version mentioned in the requirements.

[JaneLinlalala]

I found the solution. The problem was with the optimizer. Replacing the AdamW from transformers with the pytorch one did the trick. I also made sure all the params have requires_grad to be True and no torch.no_grad() was called.

PS: I used the same transformers version mentioned in the requirements.

Hi, I want to ask that how to replacing the AdamW from transformers with the pytorch one. I can't find where the code is.
Swathikiran Sudhakaran (@swathikirans)

[swathikirans]

林嘉芊 (@JaneLinlalala) You can replace this line with from torch.optim import AdamW or alternatively downgrade transformers to 4.20.1

[JaneLinlalala]

林嘉芊 (@JaneLinlalala) You can replace this line with from torch.optim import AdamW or alternatively downgrade transformers to 4.20.1

thank you！It worked.