cake-lab / few-shot-nas Goto Github PK

python test.py --arch small
08/17 02:53:34 PM gpu device = 0
08/17 02:53:34 PM args = Namespace(data='../data', batch_size=96, lr=0.025, momentum=0.9, wd=0.0003, report_freq=50, gpu=0, epochs=600, layers=20, model_path='saved_models', auxiliary_weight=0.4, cutout=False, cutout_length=16, drop_path_prob=0.2, seed=0, arch='small', save='EXP')
08/17 02:53:37 PM param size = 3.789982MB
Files already downloaded and verified
/home/captain/.local/lib/python3.9/site-packages/torch/nn/functional.py:718: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at /pytorch/c10/core/TensorImpl.h:1156.)
return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode)
tensor(95., device='cuda:0')
Traceback (most recent call last):
File "/home/captain/competition/study/Few-shot-NAS/Few-Shot-NAS_cifar10/test.py", line 134, in
main()
File "/home/captain/competition/study/Few-shot-NAS/Few-Shot-NAS_cifar10/test.py", line 97, in main
valid_acc, valid_obj = infer(valid_queue, model, criterion)
File "/home/captain/competition/study/Few-shot-NAS/Few-Shot-NAS_cifar10/test.py", line 117, in infer
prec1, prec5 = utils.accuracy(logits, target, topk=(1, 5))
File "/home/captain/competition/study/Few-shot-NAS/Few-Shot-NAS_cifar10/utils.py", line 43, in accuracy
correct_k = correct[:k].view(-1).float().sum(0)
RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.

Really appreciate your great work and I have some questions when reproducing your work.

I reproduced the ranking consistency results of One-shot NAS with kendall's Tau of 0.5438, but failed to reproduce the results of few-shot nas. I got only 0.018 kendall's tau on few-shot nas.

Here is my scripts for running few-shot nas:

1. train the few-shot supernets

bash ./supernet/few-shot/train.sh cifar10 0 ../TORCH_HOME seed-0-last-info.pth 4

2. evaluate the few-shot nas

bash ./supernet/few-shot/eval.sh cifar10 0 ../TORCH_HOME ./supernet_info 

3. get the rank consistency

python rank.py 

and we got:

Here are parts of log when evaluating few-shot nas:

I would be grateful if you can help me with the reproduction problem. 😄