loss does not have a grad fn about i-code HOT 5 CLOSED

Yea a simple solution could be:
for param in model.parameters():
param.requires_grad = True
And maybe also check the training tool you use if there is any with torch.no_grad(), etc.

from i-code.

I found the solution. The problem was with the optimizer. Replacing the AdamW from transformers with the pytorch one did the trick. I also made sure all the params have requires_grad to be True and no torch.no_grad() was called.

PS: I used the same transformers version mentioned in the requirements.

from i-code.

I found the solution. The problem was with the optimizer. Replacing the AdamW from transformers with the pytorch one did the trick. I also made sure all the params have requires_grad to be True and no torch.no_grad() was called.

PS: I used the same transformers version mentioned in the requirements.

Hi, I want to ask that how to replacing the AdamW from transformers with the pytorch one. I can't find where the code is.
@swathikirans

from i-code.

@JaneLinlalala You can replace this line with from torch.optim import AdamW or alternatively downgrade transformers to 4.20.1

from i-code.

@JaneLinlalala You can replace this line with from torch.optim import AdamW or alternatively downgrade transformers to 4.20.1

thank you！It worked.

from i-code.