Depereciation warning in pytorch 1.5 (maybe and above?) about adamp HOT 4 CLOSED

Hi

We implement AdamP based on PyTorch 1.3
PyTorch 1.5 changes default args of in-place operations such as add_() and addcmul_().
So, following lines of AdamP makes the warnings

If you change it as

 exp_avg.mul_(beta1).add_(grad, alpha=1 - beta1) 
 exp_avg_sq.mul_(beta2).addcmul_(grad, grad, value=1 - beta2) 

 p.data.add_(perturb, alpha=-step_size) 

then the warnings will disappear
But, I'm not sure whether this changes is compatible with old versions. So, I will not apply the changes to master or pip.

You can also find difference in official AdamW.py
Torch 1.4
https://github.com/pytorch/pytorch/blob/74044638f755cd8667bedc73da4dbda4aa64c948/torch/optim/adamw.py#L100-L101
Torch 1.5
https://github.com/pytorch/pytorch/blob/4ff3872a2099993bf7e8c588f7182f3df777205b/torch/optim/adamw.py#L104-L105

from adamp.

@Junyoungpark
We just release adamp==0.3.0(https://github.com/clovaai/AdamP/releases/tag/v0.3.0).
Please retry pip install adamp --upgrade :)

from adamp.

thanks for the comments! I'll use the code snippet for torch 1.5

from adamp.

Inplace operation was changed in #7.
Thanks for your contribution.

from adamp.