Question about search on custom dataset about pc-darts HOT 5 OPEN

Hi, @KleinXin , thanks for your interest of our work. As you mentioned that it is a different dataset and task. the hyperparamters need to be consistent with your usual hyper parameters in your task which may have some difference with the training on imagenet. Besides, the accuracy of the search phase is not informative enough to judge the goodness of the searched architectures. You need to train from scratch with the final searched architectures. Besides, the stop epoch is also changeable.

from pc-darts.

Hi, @KleinXin , thanks for your interest of our work. As you mentioned that it is a different dataset and task. the hyperparamters need to be consistent with your usual hyper parameters in your task which may have some difference with the training on imagenet. Besides, the accuracy of the search phase is not informative enough to judge the goodness of the searched architectures. You need to train from scratch with the final searched architectures. Besides, the stop epoch is also changeable.

Do you think it is possible to complete searching and training in one-shot?
In fact, now we are trying to finish the recognition job in the searching step instead of searching architecture followed by training model.

from pc-darts.

Another question is that in 'train_search_imagenet.py' line 178, the parameter 'architect' is not transferred into train function as in 'train_searcch.py'.
In line 206 of 'train_search_imagenet.py', architect.step is commented.

Does this mean the architecture is not updated in 'train_search_imagenet.py' ？

from pc-darts.

@KleinXin, in fact the architecture is updated. I just moved the update codes from architecture.py towards this file. The update code is the corresponding lines: 200~205. Hyperpaprameter args.begin controls from which epoch the arch paras begin to update. Besides, you can double check by see the printed arch para.

from pc-darts.

Hi, @KleinXin , thanks for your interest of our work. As you mentioned that it is a different dataset and task. the hyperparamters need to be consistent with your usual hyper parameters in your task which may have some difference with the training on imagenet. Besides, the accuracy of the search phase is not informative enough to judge the goodness of the searched architectures. You need to train from scratch with the final searched architectures. Besides, the stop epoch is also changeable.

Thanks for your kind reply, really helpful.

I came across the same problem when applying to another classification task, and I just wonder do you meet similar problem that accuracy significantly decreases after epoch args.begin, when searching on CIFAR10 or ImageNet?

And I am interested in your log files when searching on CIFAR10 or ImageNet, just for reference. Would you like to share it ?

from pc-darts.