mary-phuong / multiexit-distillation Goto Github PK

Hi Mary,

Thanks a ton for the excellent implementation. It has been really helpful in my own work. I was wondering if you could please share the pre-trained models for ImageNet (or the code). In case you need to email me my address is: [email protected]

hello. l am a college student studying deep learning in Korea.

i read your paper impressed.

i was wondering while reading the paper. did you scale each loss (depending on network depth) when configuring a multi-exit loss?

for example, ( depth = exit1 < exit2 < exit3)

exit1 , exit2, exit3 = model(input)

loss1 = criterion(exit1, targets)
loss2 = criterion(exit2, targets)
loss3 = criterion(exit3, targets)

no scale

total_loss = loss1 + loss2 + loss3

scale

total_loss = 0.1loss1 + 0.2loss2 + 0.7*loss3

If you didn't scale, can you tell me why?

i wanted to solve it by myself, but i can't solve it.
i'm really sorry.

best regards

Hi Phuong,

Could you please explain what is global_scale parameter is and how did you choose the particular values i.e, 2.0 and 5 ? in global_scale = 2.0 * 5/cf_net['n_exits']

Hi. I am a student in Korea. I was impressed with your iccv2019 announcement.

if you don't mind , I have a one question. I wonder if you modify your KLD loss like this. It's a bit hard to understand the code, so I wonder if it's different from the existing KLD loss.

KD_loss = nn.KLDivLoss()(F.log_softmax(outputs/T, dim=1), F.softmax(teacher_outputs/T, dim=1)) * (alpha * T * T) + \ F.cross_entropy(outputs, labels) * (1. - alpha)

https://github.com/mary-phuong/multiexit/distillation/blob/faff910ebdcf37c530f5bc419f31d6f587ca1545/main.py#L721

Hi Mary,

Could you please share the pre-trained models for ImageNet?
Thanks!

Hey Mary,
I am getting some run errors, it would be helpful if you could share the version of your operating system. I tried to run it on ubuntu 16.04 and 20.04. Thank you.

show error outof index :
line 41, in train
net = get_net()
line 21, in get_net
net = getattr(main, cf_net['call'])(**dict_drop(cf_net, 'call'))
line 928, in init
in_shape if i == 0 else 0, btneck_widths)