Could you pls release the search code for ImageNet about pc-darts HOT 14 CLOSED

Hi, @iamweiweishi , thanks for pointing out this type, you are correct. I have update the code. If there were other types please let me know. Thanks again! You mentioned three typos in the attached picture and what is the third one (the third red circle)?

from pc-darts.

Hi, @iamweiweishi , thanks for your interest. We will release the code on imagenet recently and the core part of the code has already been released as model_search_imagenet.py. Because of the original main code is running in the company with many specialized dependencies packet. I need more time to clear the code for public while I will try my best.

from pc-darts.

Hello, @yuhuixu1993 I am also looking for the ImageNet search code. Could you please update us on this?
Thank you

from pc-darts.

@iamweiweishi ,@carolinadp , we have uploaded the ImageNet search code.

from pc-darts.

Thank you.
But I found some typo in the 'train_search_imagenet.py' from line 121 to line 130.
The current_lr is not defined.

Shoud I modify the code like this:

And I found that the 'architect.step' is not included in 'train_search_imagenet.py', I am not sure if this is correct?
Best wishes.

from pc-darts.

@yuhuixu1993
Hi, I also swap the order of parameters 'optimizer_a' and 'criterion', in the function 'train'. The code in Line 137 could work now.
Thank you again.

from pc-darts.

@iamweiweishi, I also updated the code. Thanks for your help!

from pc-darts.

Hi, there
I have tried several times to reproduce the results, but never succeed.
The acc seldom surpassed 0.10.
I also noticed that the arch_learning_rate was never used.
Could you pls post some advice on this?
Thanks.

The acc goes like this,
12/31 07:54:28 AM train 000 6.906188e+00 0.195312 0.585938
12/31 07:59:24 AM train 050 6.910337e+00 0.091912 0.547641
12/31 08:04:19 AM train 100 6.910375e+00 0.098623 0.547262
12/31 08:09:14 AM train 150 6.910520e+00 0.094423 0.544547
12/31 08:14:08 AM train 200 6.910999e+00 0.100086 0.558730
12/31 08:18:59 AM train 250 6.911148e+00 0.102291 0.555182

from pc-darts.

@iamweiweishi, could you please offer the full log file? Thanks.

from pc-darts.

The logs:

2019-12-30 20:37:15,826 args = Namespace(arch_learning_rate=0.006, arch_weight_decay=0.001, batch_size=512, begin=35, cutout=False, cutout_length=16, drop_path_prob=0.3, epochs=50, grad_clip=5, init_channels=16, layers=8, learning_rate=0.5, learning_rate_min=0.0, model_path='saved_models', momentum=0.9, note='try', report_freq=50, save='search-log-20191230-203714', seed=2, tmp_data_dir='/home/s00444365/datasets', unrolled=False, weight_decay=0.0003, workers=32)
2019-12-30 20:37:53,315 param size = 0.956512MB
2019-12-30 20:37:54,209 epoch 0 lr 5.000000e-01
2019-12-30 20:37:55,624 genotype = Genotype(normal=[('sep_conv_5x5', 1), ('sep_conv_3x3', 0), ('dil_conv_5x5', 0), ('max_pool_3x3', 1), ('dil_conv_5x5', 0), ('max_pool_3x3', 3), ('dil_conv_5x5', 4), ('max_pool_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_5x5', 1), ('sep_conv_5x5', 0), ('max_pool_3x3', 0), ('avg_pool_3x3', 2), ('sep_conv_3x3', 2), ('avg_pool_3x3', 1), ('sep_conv_5x5', 2), ('max_pool_3x3', 1)], reduce_concat=range(2, 6))
2019-12-30 21:04:07,088 train 000 6.915554e+00 0.000000 0.390625
2019-12-30 21:28:35,755 train 050 6.912632e+00 0.103401 0.497855
2019-12-30 21:33:25,617 train 100 6.912803e+00 0.110226 0.525990
2019-12-30 21:38:14,259 train 150 6.912623e+00 0.103477 0.529025
2019-12-30 21:43:03,691 train 200 6.912730e+00 0.107859 0.550956
2019-12-30 21:53:15,624 train 250 6.912362e+00 0.114004 0.562210
2019-12-30 21:53:16,850 train_acc 0.114004
2019-12-30 21:53:17,205 epoch 1 lr 4.995067e-01
2019-12-30 21:53:17,208 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('skip_connect', 2), ('dil_conv_3x3', 1), ('sep_conv_3x3', 3), ('dil_conv_3x3', 2), ('sep_conv_3x3', 3), ('dil_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('avg_pool_3x3', 1), ('avg_pool_3x3', 0), ('sep_conv_3x3', 1), ('avg_pool_3x3', 2), ('dil_conv_3x3', 0), ('dil_conv_5x5', 4), ('sep_conv_5x5', 0)], reduce_concat=range(2, 6))
2019-12-30 22:15:07,180 train 000 6.912509e+00 0.000000 0.000000
2019-12-30 22:20:10,796 train 050 6.911884e+00 0.103401 0.566789
2019-12-30 22:25:18,731 train 100 6.911668e+00 0.104425 0.535659
2019-12-30 22:30:19,724 train 150 6.911179e+00 0.107357 0.561362
2019-12-30 22:35:14,149 train 200 6.911370e+00 0.110774 0.562617
2019-12-30 22:40:02,210 train 250 6.911471e+00 0.103853 0.555963
2019-12-30 22:40:03,586 train_acc 0.103853
2019-12-30 22:40:03,971 epoch 2 lr 4.980287e-01
2019-12-30 22:40:03,974 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('avg_pool_3x3', 1), ('dil_conv_5x5', 1), ('sep_conv_3x3', 0), ('avg_pool_3x3', 2), ('dil_conv_5x5', 0), ('dil_conv_5x5', 4), ('max_pool_3x3', 0)], reduce_concat=range(2, 6))
2019-12-30 22:40:42,512 train 000 6.910872e+00 0.390625 0.585938
2019-12-30 22:45:33,480 train 050 6.911595e+00 0.149357 0.643382
2019-12-30 22:50:29,409 train 100 6.911848e+00 0.117961 0.574335
2019-12-30 22:55:22,001 train 150 6.911471e+00 0.117705 0.576883
2019-12-30 23:00:12,741 train 200 6.911465e+00 0.117576 0.583022
2019-12-30 23:05:01,275 train 250 6.911376e+00 0.113223 0.591882
2019-12-30 23:05:02,763 train_acc 0.113223
2019-12-30 23:05:03,084 epoch 3 lr 4.955718e-01
2019-12-30 23:05:03,086 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('skip_connect', 2), ('dil_conv_3x3', 1), ('sep_conv_3x3', 3), ('dil_conv_3x3', 2), ('sep_conv_3x3', 2), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('avg_pool_3x3', 1), ('dil_conv_3x3', 1), ('avg_pool_3x3', 0), ('avg_pool_3x3', 0), ('dil_conv_3x3', 2), ('sep_conv_5x5', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6))
2019-12-30 23:05:48,591 train 000 6.917373e+00 0.390625 0.976562
2019-12-30 23:10:40,070 train 050 6.912191e+00 0.099571 0.551471
2019-12-30 23:15:33,585 train 100 6.911378e+00 0.123762 0.587871
2019-12-30 23:20:28,940 train 150 6.911189e+00 0.113825 0.579470
2019-12-30 23:25:21,135 train 200 6.911420e+00 0.103972 0.556786
2019-12-30 23:30:10,627 train 250 6.911249e+00 0.104634 0.555182
2019-12-30 23:30:12,118 train_acc 0.104634
2019-12-30 23:30:12,447 epoch 4 lr 4.921458e-01
2019-12-30 23:30:12,450 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_5x5', 2), ('dil_conv_3x3', 1), ('sep_conv_3x3', 3), ('dil_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_5x5', 1), ('dil_conv_5x5', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 0), ('dil_conv_5x5', 0), ('avg_pool_3x3', 2), ('dil_conv_5x5', 4), ('sep_conv_5x5', 0)], reduce_concat=range(2, 6))
2019-12-30 23:30:55,789 train 000 6.902474e+00 0.000000 0.976562
2019-12-30 23:35:48,190 train 050 6.910361e+00 0.088082 0.547641
2019-12-30 23:40:40,423 train 100 6.910512e+00 0.098623 0.537593
2019-12-30 23:45:35,107 train 150 6.910753e+00 0.104770 0.514797
2019-12-30 23:50:27,613 train 200 6.910940e+00 0.105916 0.511116
2019-12-30 23:55:16,571 train 250 6.910860e+00 0.106195 0.536442
2019-12-30 23:55:17,866 train_acc 0.106195
2019-12-30 23:55:18,187 epoch 5 lr 4.877641e-01
2019-12-30 23:55:18,189 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('skip_connect', 2), ('dil_conv_3x3', 1), ('sep_conv_3x3', 3), ('sep_conv_5x5', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 1), ('avg_pool_3x3', 0), ('sep_conv_3x3', 0), ('avg_pool_3x3', 2), ('sep_conv_5x5', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6))
2019-12-30 23:56:00,794 train 000 6.908577e+00 0.195312 0.195312
2019-12-31 00:00:51,679 train 050 6.910772e+00 0.045956 0.562960
2019-12-31 00:05:44,815 train 100 6.911373e+00 0.085087 0.605275
2019-12-31 00:10:36,682 train 150 6.910863e+00 0.087955 0.587231
2019-12-31 00:15:28,693 train 200 6.910745e+00 0.089397 0.604400
2019-12-31 00:20:19,079 train 250 6.910781e+00 0.090578 0.590321
2019-12-31 00:20:20,461 train_acc 0.090578
2019-12-31 00:20:20,772 epoch 6 lr 4.824441e-01
2019-12-31 00:20:20,774 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 2), ('dil_conv_3x3', 1), ('sep_conv_3x3', 3), ('dil_conv_3x3', 2), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('sep_conv_5x5', 0), ('avg_pool_3x3', 0), ('avg_pool_3x3', 2), ('sep_conv_5x5', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6))
2019-12-31 00:20:57,925 train 000 6.904783e+00 0.195312 0.195312
2019-12-31 00:25:49,618 train 050 6.912709e+00 0.114890 0.536152
2019-12-31 00:30:43,836 train 100 6.911638e+00 0.116027 0.562732
2019-12-31 00:35:36,242 train 150 6.911309e+00 0.116411 0.578177
2019-12-31 00:40:26,974 train 200 6.911143e+00 0.103972 0.562617
2019-12-31 00:45:17,071 train 250 6.910941e+00 0.101510 0.559087
2019-12-31 00:45:18,610 train_acc 0.101510
2019-12-31 00:45:19,021 epoch 7 lr 4.762068e-01
2019-12-31 00:45:19,024 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('dil_conv_3x3', 1), ('sep_conv_3x3', 3), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0), ('max_pool_3x3', 2)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('avg_pool_3x3', 1), ('sep_conv_3x3', 1), ('avg_pool_3x3', 0), ('dil_conv_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_5x5', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6))
2019-12-31 00:46:00,966 train 000 6.904390e+00 0.195312 0.781250
2019-12-31 00:50:55,475 train 050 6.909521e+00 0.130208 0.562960
2019-12-31 00:55:49,513 train 100 6.910160e+00 0.135365 0.572401
2019-12-31 01:00:45,110 train 150 6.910941e+00 0.125466 0.565242
2019-12-31 01:05:40,730 train 200 6.911049e+00 0.117576 0.583994
2019-12-31 01:10:30,990 train 250 6.911116e+00 0.114004 0.580170
2019-12-31 01:10:32,439 train_acc 0.114004
2019-12-31 01:10:32,822 epoch 8 lr 4.690767e-01
2019-12-31 01:10:32,824 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_3x3', 2), ('dil_conv_3x3', 1), ('dil_conv_3x3', 3), ('sep_conv_3x3', 2), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('avg_pool_3x3', 1), ('avg_pool_3x3', 1), ('avg_pool_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2), ('skip_connect', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6))
2019-12-31 01:11:14,794 train 000 6.913342e+00 0.000000 0.390625
2019-12-31 01:16:08,157 train 050 6.911868e+00 0.111060 0.520833
2019-12-31 01:21:04,022 train 100 6.911707e+00 0.116027 0.572401
2019-12-31 01:25:58,755 train 150 6.911452e+00 0.112531 0.563949
2019-12-31 01:30:53,288 train 200 6.911111e+00 0.113689 0.555815
2019-12-31 01:35:41,603 train 250 6.910989e+00 0.115565 0.554402
2019-12-31 01:35:42,874 train_acc 0.115565
2019-12-31 01:35:43,264 epoch 9 lr 4.610820e-01
2019-12-31 01:35:43,266 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_3x3', 2), ('dil_conv_3x3', 1), ('sep_conv_3x3', 3), ('dil_conv_5x5', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('avg_pool_3x3', 1), ('avg_pool_3x3', 0), ('dil_conv_3x3', 1), ('avg_pool_3x3', 0), ('sep_conv_3x3', 2), ('dil_conv_3x3', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6))
2019-12-31 01:36:20,137 train 000 6.916134e+00 0.000000 0.390625
2019-12-31 01:41:19,080 train 050 6.910622e+00 0.061275 0.517004
2019-12-31 01:46:16,756 train 100 6.910502e+00 0.083153 0.524056
2019-12-31 01:51:17,340 train 150 6.910645e+00 0.089249 0.519971
2019-12-31 01:56:13,444 train 200 6.910736e+00 0.091340 0.521805
2019-12-31 02:01:01,445 train 250 6.911053e+00 0.095263 0.538004
2019-12-31 02:01:02,883 train_acc 0.095263
2019-12-31 02:01:03,283 epoch 10 lr 4.522542e-01
2019-12-31 02:01:03,285 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('skip_connect', 2), ('dil_conv_3x3', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('sep_conv_3x3', 1), ('avg_pool_3x3', 0), ('dil_conv_3x3', 1), ('avg_pool_3x3', 0), ('sep_conv_5x5', 2), ('dil_conv_3x3', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6))
2019-12-31 02:01:39,403 train 000 6.900641e+00 0.195312 0.976562
2019-12-31 02:06:38,031 train 050 6.908765e+00 0.111060 0.555300
2019-12-31 02:11:31,900 train 100 6.910181e+00 0.108292 0.529858
2019-12-31 02:16:27,253 train 150 6.910482e+00 0.093129 0.551014
2019-12-31 02:21:19,424 train 200 6.910984e+00 0.097170 0.541239
2019-12-31 02:26:09,641 train 250 6.910902e+00 0.092140 0.534100
2019-12-31 02:26:11,150 train_acc 0.092140
2019-12-31 02:26:11,467 epoch 11 lr 4.426283e-01
2019-12-31 02:26:11,469 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('skip_connect', 2), ('dil_conv_3x3', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('avg_pool_3x3', 1), ('avg_pool_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 0), ('sep_conv_5x5', 2), ('sep_conv_5x5', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6))
2019-12-31 02:26:48,175 train 000 6.903584e+00 0.000000 0.390625
2019-12-31 02:31:43,546 train 050 6.910579e+00 0.130208 0.585938
2019-12-31 02:36:39,948 train 100 6.911383e+00 0.112160 0.578202
2019-12-31 02:41:35,999 train 150 6.911111e+00 0.102183 0.570416
2019-12-31 02:46:29,763 train 200 6.911105e+00 0.104944 0.572334
2019-12-31 02:51:18,680 train 250 6.911020e+00 0.109319 0.561429
2019-12-31 02:51:19,927 train_acc 0.109319
2019-12-31 02:51:20,247 epoch 12 lr 4.322422e-01
2019-12-31 02:51:20,249 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('avg_pool_3x3', 1), ('sep_conv_5x5', 2), ('dil_conv_3x3', 1), ('dil_conv_3x3', 3), ('dil_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('sep_conv_3x3', 1), ('avg_pool_3x3', 0), ('dil_conv_5x5', 1), ('avg_pool_3x3', 0), ('avg_pool_3x3', 2), ('sep_conv_5x5', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6))
2019-12-31 02:52:04,109 train 000 6.916040e+00 0.000000 0.585938
2019-12-31 02:57:01,260 train 050 6.911184e+00 0.134038 0.635723
2019-12-31 03:01:56,154 train 100 6.910871e+00 0.106358 0.607209
2019-12-31 03:06:51,014 train 150 6.911156e+00 0.104770 0.591111
2019-12-31 03:11:44,898 train 200 6.911006e+00 0.113689 0.592739
2019-12-31 03:16:34,552 train 250 6.911048e+00 0.114004 0.577046
2019-12-31 03:16:35,996 train_acc 0.114004
2019-12-31 03:16:36,411 epoch 13 lr 4.211368e-01
2019-12-31 03:16:36,413 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_5x5', 2), ('dil_conv_3x3', 1), ('sep_conv_3x3', 3), ('max_pool_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 0), ('avg_pool_3x3', 1), ('dil_conv_3x3', 1), ('avg_pool_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_5x5', 2), ('sep_conv_5x5', 0), ('avg_pool_3x3', 4)], reduce_concat=range(2, 6))
2019-12-31 03:17:19,532 train 000 6.916420e+00 0.000000 0.195312
2019-12-31 03:22:13,422 train 050 6.910109e+00 0.099571 0.486366
2019-12-31 03:27:08,115 train 100 6.910328e+00 0.088954 0.506652
2019-12-31 03:32:04,616 train 150 6.910851e+00 0.106064 0.518678
2019-12-31 03:37:00,820 train 200 6.911063e+00 0.112718 0.531522
2019-12-31 03:41:50,908 train 250 6.911088e+00 0.105414 0.524729
2019-12-31 03:41:52,458 train_acc 0.105414
2019-12-31 03:41:52,776 epoch 14 lr 4.093560e-01
2019-12-31 03:41:52,779 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_3x3', 2), ('dil_conv_3x3', 1), ('sep_conv_3x3', 3), ('dil_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 0), ('avg_pool_3x3', 1), ('avg_pool_3x3', 0), ('dil_conv_3x3', 1), ('dil_conv_5x5', 0), ('sep_conv_3x3', 2), ('sep_conv_5x5', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6))
2019-12-31 03:42:33,819 train 000 6.912027e+00 0.000000 0.585938
2019-12-31 03:47:28,531 train 050 6.911236e+00 0.137868 0.597426
2019-12-31 03:52:21,902 train 100 6.911177e+00 0.100557 0.562732
2019-12-31 03:57:14,868 train 150 6.911390e+00 0.115118 0.566536
2019-12-31 04:02:10,702 train 200 6.911063e+00 0.111746 0.564560
2019-12-31 04:06:59,930 train 250 6.911139e+00 0.105414 0.554402
2019-12-31 04:07:01,282 train_acc 0.105414
2019-12-31 04:07:01,642 epoch 15 lr 3.969463e-01
2019-12-31 04:07:01,644 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('skip_connect', 2), ('dil_conv_3x3', 1), ('dil_conv_3x3', 3), ('dil_conv_5x5', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('avg_pool_3x3', 1), ('dil_conv_5x5', 1), ('avg_pool_3x3', 0), ('dil_conv_5x5', 0), ('avg_pool_3x3', 2), ('sep_conv_5x5', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6))
2019-12-31 04:07:41,033 train 000 6.907535e+00 0.390625 0.390625
2019-12-31 04:12:37,931 train 050 6.911418e+00 0.118719 0.589767
2019-12-31 04:17:33,264 train 100 6.910759e+00 0.092822 0.568533
2019-12-31 04:22:26,657 train 150 6.911244e+00 0.098303 0.556188
2019-12-31 04:27:22,261 train 200 6.911272e+00 0.089397 0.550956
2019-12-31 04:32:11,483 train 250 6.911101e+00 0.095263 0.561429
2019-12-31 04:32:13,004 train_acc 0.095263
2019-12-31 04:32:13,312 epoch 16 lr 3.839567e-01
2019-12-31 04:32:13,314 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_5x5', 2), ('dil_conv_3x3', 1), ('sep_conv_3x3', 3), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('avg_pool_3x3', 1), ('dil_conv_5x5', 1), ('avg_pool_3x3', 0), ('avg_pool_3x3', 0), ('avg_pool_3x3', 2), ('dil_conv_5x5', 4), ('sep_conv_5x5', 3)], reduce_concat=range(2, 6))
2019-12-31 04:32:50,973 train 000 6.910200e+00 0.000000 0.390625
2019-12-31 04:37:45,113 train 050 6.911125e+00 0.118719 0.585938
2019-12-31 04:42:39,815 train 100 6.911381e+00 0.110226 0.537593
2019-12-31 04:47:35,330 train 150 6.911345e+00 0.112531 0.540666
2019-12-31 04:52:29,175 train 200 6.911152e+00 0.115633 0.535409
2019-12-31 04:57:17,484 train 250 6.911033e+00 0.115565 0.548155
2019-12-31 04:57:18,993 train_acc 0.115565
2019-12-31 04:57:19,319 epoch 17 lr 3.704384e-01
2019-12-31 04:57:19,321 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_5x5', 2), ('dil_conv_3x3', 1), ('dil_conv_3x3', 3), ('dil_conv_5x5', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 1), ('avg_pool_3x3', 0), ('dil_conv_5x5', 0), ('avg_pool_3x3', 2), ('dil_conv_5x5', 4), ('sep_conv_3x3', 0)], reduce_concat=range(2, 6))
2019-12-31 04:58:04,018 train 000 6.912227e+00 0.000000 0.390625
2019-12-31 05:03:01,127 train 050 6.911225e+00 0.103401 0.628064
2019-12-31 05:07:53,678 train 100 6.911846e+00 0.092822 0.554997
2019-12-31 05:12:46,673 train 150 6.911492e+00 0.087955 0.525145
2019-12-31 05:17:38,750 train 200 6.911056e+00 0.088425 0.525692
2019-12-31 05:22:30,302 train 250 6.910874e+00 0.095263 0.527072
2019-12-31 05:22:31,557 train_acc 0.095263
2019-12-31 05:22:31,867 epoch 18 lr 3.564448e-01
2019-12-31 05:22:31,869 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_5x5', 2), ('dil_conv_3x3', 1), ('sep_conv_3x3', 3), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('avg_pool_3x3', 1), ('sep_conv_3x3', 1), ('avg_pool_3x3', 0), ('avg_pool_3x3', 0), ('avg_pool_3x3', 2), ('sep_conv_5x5', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6))
2019-12-31 05:23:16,157 train 000 6.913686e+00 0.585938 0.781250
2019-12-31 05:28:09,088 train 050 6.910643e+00 0.114890 0.555300
2019-12-31 05:33:04,475 train 100 6.910024e+00 0.094756 0.524056
2019-12-31 05:37:59,845 train 150 6.910359e+00 0.109944 0.548427
2019-12-31 05:42:53,748 train 200 6.910761e+00 0.106887 0.563588
2019-12-31 05:47:44,195 train 250 6.910785e+00 0.108538 0.556744
2019-12-31 05:47:45,677 train_acc 0.108538
2019-12-31 05:47:45,986 epoch 19 lr 3.420311e-01
2019-12-31 05:47:45,988 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('skip_connect', 2), ('dil_conv_3x3', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('avg_pool_3x3', 1), ('dil_conv_5x5', 1), ('avg_pool_3x3', 0), ('dil_conv_5x5', 0), ('sep_conv_5x5', 2), ('sep_conv_3x3', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6))
2019-12-31 05:48:26,255 train 000 6.913815e+00 0.000000 0.390625
2019-12-31 05:53:24,603 train 050 6.910912e+00 0.080423 0.532322
2019-12-31 05:58:18,362 train 100 6.910938e+00 0.114093 0.576269
2019-12-31 06:03:15,480 train 150 6.911035e+00 0.117705 0.582057
2019-12-31 06:08:09,036 train 200 6.910852e+00 0.118548 0.592739
2019-12-31 06:12:59,343 train 250 6.910917e+00 0.114004 0.578608
2019-12-31 06:13:00,873 train_acc 0.114004
2019-12-31 06:13:01,195 epoch 20 lr 3.272542e-01
2019-12-31 06:13:01,197 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 2), ('dil_conv_3x3', 1), ('sep_conv_3x3', 3), ('dil_conv_5x5', 2), ('sep_conv_3x3', 0), ('sep_conv_3x3', 2)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 1), ('avg_pool_3x3', 0), ('avg_pool_3x3', 1), ('avg_pool_3x3', 0), ('avg_pool_3x3', 0), ('sep_conv_3x3', 2), ('sep_conv_5x5', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6))
2019-12-31 06:13:39,429 train 000 6.912564e+00 0.195312 0.976562
2019-12-31 06:18:34,403 train 050 6.909996e+00 0.118719 0.505515
2019-12-31 06:23:28,270 train 100 6.910769e+00 0.102491 0.531791
2019-12-31 06:28:24,820 train 150 6.910868e+00 0.099596 0.548427
2019-12-31 06:33:20,010 train 200 6.911206e+00 0.097170 0.553871
2019-12-31 06:38:11,998 train 250 6.911031e+00 0.094483 0.544251
2019-12-31 06:38:13,448 train_acc 0.094483
2019-12-31 06:38:13,941 epoch 21 lr 3.121725e-01
2019-12-31 06:38:13,944 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('skip_connect', 2), ('dil_conv_3x3', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('avg_pool_3x3', 1), ('dil_conv_5x5', 1), ('avg_pool_3x3', 0), ('sep_conv_3x3', 0), ('sep_conv_5x5', 2), ('skip_connect', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6))
2019-12-31 06:38:53,487 train 000 6.916465e+00 0.000000 0.585938
2019-12-31 06:43:50,193 train 050 6.911452e+00 0.130208 0.559130
2019-12-31 06:48:43,430 train 100 6.910875e+00 0.121829 0.566600
2019-12-31 06:53:39,160 train 150 6.911049e+00 0.117705 0.560068
2019-12-31 06:58:32,210 train 200 6.911103e+00 0.111746 0.568447
2019-12-31 07:03:19,900 train 250 6.910911e+00 0.103853 0.570799
2019-12-31 07:03:21,413 train_acc 0.103853
2019-12-31 07:03:21,742 epoch 22 lr 2.968453e-01
2019-12-31 07:03:21,744 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 2), ('dil_conv_3x3', 1), ('sep_conv_3x3', 3), ('dil_conv_3x3', 2), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('dil_conv_5x5', 1), ('avg_pool_3x3', 0), ('dil_conv_5x5', 1), ('avg_pool_3x3', 0), ('avg_pool_3x3', 2), ('sep_conv_3x3', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6))
2019-12-31 07:03:58,330 train 000 6.918197e+00 0.000000 0.390625
2019-12-31 07:08:50,771 train 050 6.910912e+00 0.095741 0.570619
2019-12-31 07:13:46,871 train 100 6.911359e+00 0.090888 0.541460
2019-12-31 07:18:40,065 train 150 6.910955e+00 0.099596 0.578177
2019-12-31 07:23:32,419 train 200 6.911001e+00 0.100086 0.571362
2019-12-31 07:28:21,888 train 250 6.911139e+00 0.098387 0.560648
2019-12-31 07:28:23,402 train_acc 0.098387
2019-12-31 07:28:23,711 epoch 23 lr 2.813333e-01
2019-12-31 07:28:23,713 genotype = Genotype(normal=[('sep_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 2), ('dil_conv_3x3', 1), ('sep_conv_3x3', 3), ('dil_conv_5x5', 2), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0), ('avg_pool_3x3', 1), ('dil_conv_5x5', 1), ('avg_pool_3x3', 0), ('dil_conv_5x5', 0), ('avg_pool_3x3', 2), ('sep_conv_3x3', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6))
2019-12-31 07:29:07,046 train 000 6.917174e+00 0.000000 0.585938
2019-12-31 07:34:05,083 train 050 6.910776e+00 0.076593 0.536152
2019-12-31 07:39:01,895 train 100 6.911441e+00 0.083153 0.533725
2019-12-31 07:43:57,263 train 150 6.911437e+00 0.086662 0.540666
2019-12-31 07:48:53,471 train 200 6.911353e+00 0.088425 0.551928
2019-12-31 07:53:43,070 train 250 6.911177e+00 0.089797 0.552059
2019-12-31 07:53:44,538 train_acc 0.089797
2019-12-31 07:53:44,851 epoch 24 lr 2.656976e-01

Thank you. @yuhuixu1993

from pc-darts.

@iamweiweishi ，hi，first you need to check the warm up code （the lr in the first five epochs）The log shows that you did not warm
up

Second，the archtect learning rate didnot work until the 35th epoch

Third， you need to check that the data is ramdomly sampled in each class of the1000 classes

from pc-darts.

I have also tried the warmup training. The results remained the same.
The data are randomly sampled as you metioned.

I rerun the code again, it seems work well now although I still did not find what the problem is.
Many thanks. ^^

from pc-darts.

@iamweiweishi ，I will try it again myself，does the archtecture learning rate work after 35th epoch？

from pc-darts.

@iamweiweishi , there indeed some problems in the the previous train_search_imagenet.py. I have update the code I have tested ok.

from pc-darts.