zheng-ningxin / brp-nas Goto Github PK

Hello! Thank you very much for such an excellent work. I have a question. How did the spearman in Figure 5 in the paper get it through the binary relation preditor? As far as I know, spearman is a statistic that requires global sorting, but the binary relation preditor can only give the relationship between two models

.

Hello,

I was trying to run the run.sh file by default, I used the examples/ folder as specified. However, I noticed that the data-set is encoded in the .pickle file as:
{(3, 4, 4, 0, 3, 0): 0.00520570158958435, (4, 4, 4, 0, 3, 0): 0.005266194343566895}

        def transform_to_pairs(triplets):
            return [[[t[0], t[1]], t[2]] for t in triplets]

This is incompatible with the parsing code above, as the NASBench201 architecutres are not stored in the way that you intend.

Further, in eagle/predictors/main.py

            for k in dataset.keys():
                if any([k.startswith(n) for n in subset_name]):
                    result.append(dataset[k])

I do not think this will work, because the pickle files that are provided are only for nasbench201, maybe this was done with future benchmark inclusions in mind, but it breaks currently.

Further, what is ' /home/core/znx/gcn_latency/Li_latency_data_vpu.pickle'? This is present in both .sh files, but I cannot find them.

Additionally, is there a reason why --measurement and --dataset_path both have to be specified? I do not know if its evident from the documentation, might be something obvious i am missing..

Thanks for the help! :)