When training with different budgets in adversary/train.py, you do not reset the model each run. So if you want to train with a budget of 100 and then 200, you actually train with 100 and then 300 (100+200). Because the model has already seen the first 100 at the start of the second budget; the budget adds up.
model = zoo.get_net(model_name, modelfamily, pretrained, num_classes=num_classes)
model = model.to(device)
for b in budgets:
#train model
Solution: Reset or redefine the model each iteration.