Fractional Underdamped Langevin Dynamics: Retargeting SGD with Momentum under Heavy-Tailed Gradient Noise
The optimizer implementation for the paper "Fractional Underdamped Langevin Dynamics: Retargeting SGD with Momentum under Heavy-Tailed Gradient Noise", published in ICML 2020.
The class SGD_fm should be used as a pytorch optimizer. See https://pytorch.org/docs/stable/optim.html for details.
If you find this code useful, thank you for citing the following work:
@inproceedings{Simsekli2020Fractional, author = {U. Simsekli and L. Zhu and Y. W. Teh and M. Gurbuzbalaban} title = {Fractional Underdamped Langevin Dynamics: Retargeting SGD with Momentum under Heavy-Tailed Gradient Noise}, booktitle = {Proceedings of the 36th International Conference on Machine Learning, (ICML) 2020}, month = jun, year = {2020}, }