cannot reproduce the reported accuracy in the paper. about ista-nas HOT 7 OPEN

I rerun it using different seeds. The best one is 97.4%, which is still lower than 97.64%.
After all, the reported 97.64% is the best top1 in DARTS papers so far. I am eager to reproduce such a good result.

By the way, I still cannot reproduce the reported ImageNet result (76.0%) using your code (mine is 75.6%).
I would appreciate if you release the log to help me find out what's wrong.
Thanks!

from ista-nas.

Thanks for your attention. This is the training log of the experiment in our paper.
training log cifar.log

I will check the released code recently.

from ista-nas.

Thanks! This is the log file for another seed 19 (97.4%). The only difference is passing a different seed s=19
test_one_stage_s19.log

Moreover, I would appreciate if you can release the training log on ImageNet (76.0%)

from ista-nas.

One stage Imgnet resume.log
One stage Imgnet.log
One stage C10.log

Hi, the following files are some of our logs of the original evaluation on ImageNet.

I have checked the code but did not find any bug.

from ista-nas.

Thanks!
However, the remaining probability is the random seed. Can you provide more logs (different seeds) about the model searched on CIFAR10? I have run the released training script on CIFAR10 using eight seeds but none of them exceeds 97.5%.
Several classmates of mine face with the same issue.

from ista-nas.

I met the same issue as @cxxgtxy.
I ran the command for evaluating one-stage ISTA-NAS on CIFAR10 following README several times, but the accuracies were lower than 97.5%

from ista-nas.

Thanks for the great work! I tried to reproduce the accuracy that reported in the paper on CIFAR10. But I only obtained around 93-94% accuracy via running

python ./tools/evaluation.py --auxiliary --cutout --onestage --arch ISTA_onestage

Any idea how to recap the significant accuracy gap? Thanks! My experiment setting is A100 server torch 1.13.

from ista-nas.