To ensure that our experiments perform fair comparison with previous work, we mainly adopt the public codes from TRADES, and make some modifications on them to run the trials.
- Train PGD-AT
python train_adv_cifar10.py --loss=pgd
Here we activate the HE
operation and set the scale s
and margin m
.
- Train PGD-AT + HE:
python train_adv_cifar10.py --loss=pgd_he --s=10.0 --m=0.2 --wd 5e-4
- Train TRADES + HE:
python train_adv_cifar10.py --loss=trades_he --beta=6.0 --s=15.0 --m=0.2
- Train ALP + HE:
python train_adv_cifar10.py --loss=alp_he --beta=0.5 --s=20.0 --m=0.3
Our pre-trained models for CIFAR-10 WideResNets (in width factor 20 and 10) are available, including PGD+HE-wide20 and PGD+HE-wide10. We find that our HE mechanism is more suitable for the PGD-AT framework, since its principled formula is more aligned with our analysis.
To evaluate the performance of the trained models under the PGD attacks (default PGD-20), the running command is
$ python pgd_attack_cifar10.py
We also evaluate a reliable attack, named AutoAttack, the running command is
$ python test_autoattack.py --model-path=$CHECKPOINT_PATH$ --use_FNandWN --widen_factor=20
To evaluate the performance of the general robustness, we also test on the CIFAR-10-C datasets by Hendrycks & Dietterich (ICLR 2019). The running command is
$ python general_attack_cifar10.py