Comments (22)
@kunalmessi10 ,it is almost the same. accuracy during search is not that important. You may evaluate the searched aechitexture.
from pc-darts.
By same you mean same as 31%?
from pc-darts.
@kunalmessi10,yes,though not exactly the same
from pc-darts.
okay close enough is fine with me, and the searched architecture for imagenet that you report in your paper is based on random sampling of channels or channel shuffle in train_search_imagenet.py, also the architecture the search outputted didn't any skip connections, any particular reason for that?
from pc-darts.
Also, one more question, is it possible to search on a subset of imagenet which has lesser classes but the same number of images per class?
from pc-darts.
@kunalmessi10,Hi,the result is based on channel shuffle. Such situation(without skipconnect) sometimes happens. You can use the skipconnect regulation in Pdarts or search more times. We are now also working on solving the problem. Yes, I think it is ok to search on less classes.
from pc-darts.
okay, thanks!
from pc-darts.
I'm guessing Pdarts is also from your research group, can you point me out the skip connect regulation part in that paper's code?
from pc-darts.
@kunalmessi10 ,https://github.com/chenxin061/pdarts/blob/05addf3489b26edcf004fc4005bbc110b56e0075/train_search.py#L407 ,while this part of code can not be used directly. In pdarts situation,the skipconnect is too many so they delete the skip with lowest prob. While in our case it is easier,you can choose the two biggest skip in the choosed edge to replace the original operation. For example, the choose edge is (1,0) (1,2)(1,3)(1,4) in this 8 edges, you choos the biggest two skip and replace the corresponding operation. I will add this code in our further version.
from pc-darts.
from pc-darts.
Does adding a drop path after skip connect as described in pdarts helps?
…
I add skip just to follow the mobile settings(<600 flops). And skip sometimes helps in the performance of the network. The network without skip may harder to train.
from pc-darts.
Okay, thanks for the help, I'll try as you suggested
from pc-darts.
@kunalmessi10 , wish you could find you satisfied architectures. Could you please show the architecture you found? I just want to see if the code runs ok
from pc-darts.
genotype = Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('sep_conv_5x5', 1), ('sep_conv_5x5', 0), ('sep_conv_5x5', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_5x5', 4)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('dil_conv_5x5', 2), ('dil_conv_5x5', 3), ('sep_conv_3x3', 1), ('sep_conv_3x3', 4), ('sep_conv_5x5', 1)], reduce_concat=range(2, 6))
from pc-darts.
This is architecture I got
normal_alphas ---> tensor([[0.0428, 0.1214, 0.0811, 0.0891, 0.2067, 0.1785, 0.1338, 0.1466],
[0.0387, 0.1426, 0.0804, 0.1021, 0.1719, 0.1574, 0.1530, 0.1540],
[0.0546, 0.1050, 0.0971, 0.0924, 0.1706, 0.1910, 0.1396, 0.1497],
[0.0445, 0.1412, 0.1362, 0.1258, 0.1327, 0.1574, 0.1360, 0.1261],
[0.0649, 0.1036, 0.1052, 0.0952, 0.1672, 0.1853, 0.1218, 0.1568],
[0.0647, 0.1160, 0.1005, 0.0918, 0.1949, 0.1668, 0.1116, 0.1538],
[0.0577, 0.1134, 0.0930, 0.1123, 0.1778, 0.1929, 0.1167, 0.1362],
[0.0753, 0.0922, 0.0894, 0.1106, 0.1702, 0.1861, 0.1513, 0.1249],
[0.0662, 0.0930, 0.0972, 0.0918, 0.1841, 0.1787, 0.1170, 0.1720],
[0.0658, 0.1119, 0.0828, 0.0787, 0.2256, 0.2250, 0.1110, 0.0992],
[0.0671, 0.1051, 0.0855, 0.0923, 0.1946, 0.1994, 0.1088, 0.1472],
[0.0633, 0.1241, 0.1018, 0.0827, 0.2019, 0.1626, 0.1299, 0.1337],
[0.0662, 0.1256, 0.0982, 0.0829, 0.1919, 0.1486, 0.1255, 0.1612],
[0.0576, 0.1128, 0.0855, 0.0716, 0.1836, 0.1938, 0.1781, 0.1170]],
device='cuda:0', grad_fn=)
reduce_alphas ---> tensor([[0.1187, 0.1201, 0.0942, 0.1089, 0.2002, 0.1300, 0.0951, 0.1328],
[0.1623, 0.1293, 0.1215, 0.1230, 0.1330, 0.1203, 0.1079, 0.1027],
[0.1003, 0.1067, 0.0917, 0.0978, 0.2270, 0.1542, 0.0797, 0.1427],
[0.0974, 0.1663, 0.1186, 0.1336, 0.1378, 0.1487, 0.0811, 0.1165],
[0.1370, 0.1187, 0.0962, 0.1174, 0.1279, 0.1356, 0.1009, 0.1663],
[0.1018, 0.1252, 0.1185, 0.1088, 0.1425, 0.1502, 0.1512, 0.1018],
[0.1073, 0.1334, 0.1000, 0.1373, 0.1797, 0.1576, 0.0918, 0.0929],
[0.1223, 0.1598, 0.1124, 0.1403, 0.1138, 0.1180, 0.1129, 0.1204],
[0.1164, 0.1137, 0.0890, 0.1190, 0.1332, 0.1016, 0.1291, 0.1980],
[0.0977, 0.1114, 0.1015, 0.1171, 0.1864, 0.1777, 0.1089, 0.0993],
[0.0909, 0.1153, 0.0887, 0.0997, 0.1779, 0.1837, 0.1229, 0.1209],
[0.1042, 0.1514, 0.1091, 0.1285, 0.1254, 0.1521, 0.1111, 0.1183],
[0.1044, 0.1176, 0.0911, 0.1128, 0.1139, 0.1846, 0.1261, 0.1495],
[0.0951, 0.0986, 0.0768, 0.0992, 0.2042, 0.1756, 0.1103, 0.1402]],
device='cuda:0', grad_fn=)
from pc-darts.
From these alphas can you tell what would be my final arch by doing the skip connect trick you mentioned
from pc-darts.
@kunalmessi10, it seems ok, thanks.
from pc-darts.
Can you tell me the new genotypes after replacing the skip connect as you suggested? This architecture is although training fine yet
from pc-darts.
@kunalmessi10, you can check if it is right. Genotype(normal=[('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('skip_connect', 1), ('sep_conv_5x5', 0), ('skip_connect', 1), ('sep_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_5x5', 4)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 0), ('dil_conv_5x5', 2), ('dil_conv_5x5', 3), ('sep_conv_3x3', 1), ('sep_conv_3x3', 4), ('sep_conv_5x5', 1)], reduce_concat=range(2, 6))
from pc-darts.
Should we not add skip connect in the reduction cell?
from pc-darts.
@kunalmessi10, I think reduction cell is ok without skip.
from pc-darts.
okay, I'll try that
from pc-darts.
Related Issues (20)
- Is a channel sampling mask fixed? HOT 3
- Is there any plan to release the pretrained imagenet model? HOT 1
- Why modifying architecture after epoch 15
- Data preparation of ImageNet
- How to change the channel proportion K? HOT 2
- Cannot re-implement your claimed result HOT 3
- GPU Utilization is Bad HOT 1
- We cannot obtain your claimed result on ImageNet after trying many configurations HOT 4
- Question about search on custom dataset HOT 5
- test.py运行报错
- Understanding the two sets of the architecture hyperparameter HOT 2
- how you report the final accuracy in evaluation? Possibly touch the test set for the best acc? HOT 2
- Learning rate schedule
- 你好,结果不一致 HOT 2
- Searched genotype remain / keep unchanged for a great number of epoch HOT 2
- RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cuda:1!
- 您好,想请问一下网络搜索完之后如何得到需要的网络结构代码? HOT 3
- About the license of this repository
- Hello, whether PC-DARTS likes DARTS with extra dropout?
- Not Enough Comments in the Code
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pc-darts.