Although setting the same random seed, the accuracy results of two experiments with the same setting are different.

Code:
os.environ['PYTHONHASHSEED'] = str(seed)
random.seed(seed)
np.random.seed(seed)
torch.manual_seed(seed)
torch.cuda.manual_seed(seed)
torch.cuda.manual_seed_all(seed)
torch.backends.cudnn.benchmark = False
torch.backends.cudnn.deterministic = True

Looking forward to your reply!

Hi @icoz69 ,

Thanks for your great work!! I am impressed by your visualization Fig 6 & 7 in the paper to explain matching patches of images. Do you have the code or reference to reproduce this? It would be helpful if you can help. Thanks!!

Hi! Thank you for the nice work. I have a question about the gradient when using the QPTH solver to solve the LP problem. In my test, the gradients are almost always zeros. I attached the following testing code in the models/emd_utils.py file.

if __name__ == '__main__':

    batch_size = 50
    num_node = 25

    cosine_distance_matrix = torch.rand(batch_size, num_node, num_node, device='cuda', requires_grad=True)
    weight1 = torch.rand(batch_size, num_node, device='cuda', requires_grad=True)
    weight2 = torch.rand(batch_size, num_node, device='cuda', requires_grad=True)

    _, qpth_flow = emd_inference_qpth(cosine_distance_matrix, weight1, weight2)
    qpth_flow.sum().backward()
    print(weight1.grad)
    print(weight2.grad)
    print(cosine_distance_matrix.grad)

The output gradient w.r.t all three terms are very small regardless of whether L2 or QP form is used. So I am wondering how is the back-propagation done in such a condition. Would you like to tell me if there is a misunderstanding of me?

Hi,

Thanks for your inspring work! However, I have a question.

I notice that in your readme file of the project, a command is provided to meta-train the model, after pre-training is done, where opencv is specified as the solver:
-----#use opencv solver (about 8GB memory)
-----$ python train_meta.py -deepemd fcn -shot 1 -way 5 -solver opencv -gpu 0,1,2,3
Nevertheless, in your paper, 5.1 Implementation Details, Training paragraph, it's claimed that QPTH is used during meta-training: “At training time, we use the GPU accelerated convex optimization solver QPTH [76] to solve the Linear Programming problem in our network and compute gradients for back-propagation.”

So, my question is:

1. Is opencv available during meta-training?
2. If both opencv and QPTH are available during meta-training? Does that mean for opencv, additional coding work has been done in the project to calculate gradients for back-propagation, like what has been done in QPTH? I fail to find related code, if there is, could you help highlight them to me?

Best Regards,
Neil

hello, I found there was some misdescription. There are 600 samples per class on FC100, not 100 samples.

Thanks for your exciting work! When I pretrain miniimagenet , I notice that the train acc up to 92% , while the val acc only 62% , whether it has being overrfitting? or should i check my code?

Dear author,

thanks for opening source your project. I am just wondering if you can publish the training script for CUB and CIFAR datasets that can potentially achieve the results reported in your paper?

Thanks

when setname == 'val' or setname == 'test'
why?resize to 92 first?
transforms.Resize([92, 92]),
transforms.CenterCrop(84),

hi, @icoz69 , thanks for releasing the code! I am quite surprising that the results of some of the baselines are much more higher than that reported in the original paper, such as matching network, i.e., 63.08% in deepemd compared to 43% in nips paper, is it because you change the backbone to resnet12?

Looking forward to your reply.

Have you meet some problem like this?

RuntimeError: lu: For batch 0: U(30,30) is zero, singular U.

Hello, thanks for your great work first.
I downloaded the Meta-trained models and tested DeepEMD-FCN with it for 1-shot 5-way task on the miniImageNet dataset, but the result is different from that in the paper. The results of DeepEMD-FCN for 1-shot tasks on miniimagenet is 57.46, pretty lower than your experimental result. I want to know what caused it.
I am really desperate for the answer.
Thanks in advance for your help.

@icoz69 Thanks for your excellent work! When I train tieredimagenet, should I change the learning rate, max_epoch, step_size and other parameters? Or should I keep the pre-defined default parameters unchanged?

Hi. I tried to reproduce results for MiniImageNet in DeepEMD FCN following your instruction, but failed to get the accuracy listed in your paper. The evaluation accuracy is only 55%.

Here is my training scripts:

$ python train_pretrain.py -shot 1 -way 5 -dataset miniimagenet -gpu 
$ python train_meta.py -deepemd fcn -shot 1 -way 5 -solver opencv -dataset miniimagenet -gpu 0
$ python eval.py -shot 1 -way 5 -dataset miniimagenet -gpu

Could you give me some tips?
Looking forward to your reply.

Hi,

I saw the default setting (both the eval.py and the evaluation command-line script in the ReadMe file) is using opencv solver. However, for 5-shot evaluation, the structured FC weights need gradients to update. But looks like the opencv solver layer is not differentiable? So how could the SFC weights be updated given this situation?

Thanks!

-0:
1
2.....
-1:
1
2....

Thanks for your exciting work. May I ask you some questions:
Q1. When I train (meta-train) the model with 4 gpus with qpth solver, I get the RuntimeError(at the bottom). But when meta-training the model with 3 gpus with qpth solver, it is available. And it is also ok when pretraining or meta-training with opcv solver with any numbers of gpus.

Q2. What are they trying to do in the opencv solver? specifically,
1) line 5 and line 8 calculate the dot product between flow and similarity, but in transportation problem, we aim to calculate the dot product between flow and cost, so is it miss the (1-similarity_map[i, j, :, :]) in line 5? and why not directly use the cost computed by opencv?
2) what is the meaning of the temperature in line 7, and it is like that temperature is a scale factor for logitis?

  line0:     for i in range(num_query):
  line1:          for j in range(num_proto):
  line2:              # similarity, weight_1, weight_2 index [i, j]; cost = 1 - similarity
  line3:              _, flow = emd_inference_opencv(1 - similarity_map[i, j, :, :], weight_1[i, j, :], weight_2[j, i, :])
  line4:              # _: cost; flow
  line5:              similarity_map[i, j, :, :] = (similarity_map[i, j, :, :]) * torch.from_numpy(flow).cuda()
  line6:
  line7:      temperature = (self.args.temperature/num_node)
  line8:      logitis = similarity_map.sum(-1).sum(-1) * temperature      # (tensor([75,5])
  line9:      return logitis

3. why v2.EMD is much faster than qpth? do they use the different algorithms? is it necessary to use relu and normalize before EMD?

4. It is obvious that gradient doesn't bp, so do you only update structed_fc when training with opcv and the encoder is fixed?

       def emd_inference_opencv(cost_matrix, weight1, weight2):
           # (tensor([25,25]), tensor([25]), tensor([25]))
           # cost matrix is a tensor of shape [N,N]
           cost_matrix = cost_matrix.detach().cpu().numpy()
           # ensure > 0
           weight1 = F.relu(weight1) + 1e-5
           weight2 = F.relu(weight2) + 1e-5
           # normalize to 1
           weight1 = (weight1 * (weight1.shape[0] / weight1.sum().item())).view(-1, 1).detach().cpu().numpy()
           weight2 = (weight2 * (weight2.shape[0] / weight2.sum().item())).view(-1, 1).detach().cpu().numpy()
           cost, _, flow = cv2.EMD(weight1, weight2, cv2.DIST_USER, cost_matrix)
           # a transportation problem, solved using a simplex algorithm
           # the complexity is exponential in the worst case
           # multi-dimensional histogram comparison for image retrieval
           return cost, flow
   

Looking forward to your reply! Many thanks for your help!

loading model from : checkpoint/pre_train/miniimagenet/128-0.1000-30-0.20/max_acc.pth
checkpoint/miniimagenet/fcn/1shot-5way/
epo 1, total loss=0.7245 acc=0.8000: 2%|██▎ | 1/50 [00:48<39:16, 48.09s/it]
Traceback (most recent call last):
File "train_meta.py", line 157, in
logits = model((data_shot.unsqueeze(0).repeat(num_gpu, 1, 1, 1, 1), data_query))
File "/home/lichangzhen/anaconda3/envs/pytorch1.7/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/lichangzhen/anaconda3/envs/pytorch1.7/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 161, in forward
outputs = self.parallel_apply(replicas, inputs, kwargs)
File "/home/lichangzhen/anaconda3/envs/pytorch1.7/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 171, in parallel_apply
return parallel_apply(replicas, inputs, kwargs, self.device_ids[:len(replicas)])
File "/home/lichangzhen/anaconda3/envs/pytorch1.7/lib/python3.6/site-packages/torch/nn/parallel/parallel_apply.py", line 86, in parallel_apply
output.reraise()
File "/home/lichangzhen/anaconda3/envs/pytorch1.7/lib/python3.6/site-packages/torch/_utils.py", line 428, in reraise
raise self.exc_type(msg)
IndexError: Caught IndexError in replica 3 on device 3.
Original Traceback (most recent call last):
File "/home/lichangzhen/anaconda3/envs/pytorch1.7/lib/python3.6/site-packages/torch/nn/parallel/parallel_apply.py", line 61, in _worker
output = module(*input, **kwargs)
File "/home/lichangzhen/anaconda3/envs/pytorch1.7/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home1/lichangzhen/experiment/DeepEMD/Models/models/Network.py", line 26, in forward
return self.emd_forward_1shot(support, query)
File "/home1/lichangzhen/experiment/DeepEMD/Models/models/Network.py", line 75, in emd_forward_1shot
logits = self.get_emd_distance(similarity_map, weight_1, weight_2, solver='qpth')
File "/home1/lichangzhen/experiment/DeepEMD/Models/models/Network.py", line 130, in get_emd_distance
_, flows = emd_inference_qpth(1 - similarity_map, weight_1, weight_2,form=self.args.form, l2_strength=self.args.l2_strength)
File "/home1/lichangzhen/experiment/DeepEMD/Models/models/emd_utils.py", line 59, in emd_inference_qpth
flow = QPFunction(verbose=-1)(Q, p, G, h, A, b)
File "/home/lichangzhen/anaconda3/envs/pytorch1.7/lib/python3.6/site-packages/qpth/qp.py", line 96, in forward
eps, verbose, notImprovedLim, maxIter)
File "/home/lichangzhen/anaconda3/envs/pytorch1.7/lib/python3.6/site-packages/qpth/solvers/pdipm/batch.py", line 62, in forward
factor_kkt(S_LU, R, d)
File "/home/lichangzhen/anaconda3/envs/pytorch1.7/lib/python3.6/site-packages/qpth/solvers/pdipm/batch.py", line 442, in factor_kkt
T[factor_kkt_eye] += (1. / d).squeeze().view(-1)
IndexError: The shape of the mask [95, 675, 675] at index 0 does not match the shape of the indexed tensor [90, 675, 675] at index 0

Hi, thanks for the contribution to few-shot learning community. It is indeed interesting work!

I am just wondering how to get the exact results you report in the paper for tiered-imagenet. Maybe not the exact results but close results. I can get 73% accuracy on 1 shot using the hyperparameters given in the readme file with 600 randomly sampled episodes, however, only 70% with 5,000 episodes. Thanks!

I pretrained the model and then used deep emd with the default setting, but the 5-way-1-shot accuracy was just 65.5%, which was 1% lower than the paper.

@icoz69 , Does the return statement 'return x' in the encode function of class DeepEMD is unnecessary in the first two places, I think only the last one needed to be retained, what do you think?

Hello!
In the evaluation stage of minIageNet datasets, there are 20 types of samples, each of which has 600 samples, which means 12,000 samples. If you use the 5-way 1-shot experimental setting, it means that one task requires 80 images and your epoch has 5000 tasks, which means that 400,000 pictures are used, and you used data amplification? Or is the sample reused in one epoch? thank you for your patience!

Hi, thanks for your great work! I have some questions about the structure of ResNet-12 and the optimizer setting.

Firstly, why are the channels of ResNet-12 set as [64, 160, 320, 640]? Such a setting is common in WideResNet-28*10 rather than ResNet-12, which should be [64, 128, 256, 512]. Especially, in the paper, the Implementation Details section describes the feature map size which is 5*5*512. It seems a change after finishing the paper. So, I wonder how much this change has affected the performance?

Meanwhile, the optimizer adopts second-order optimization. It is also not common in the previous works. Although the training process really depends on each work, I wanna try a training from scratch without it.

Hi, I'm a beginner. I want to know why to repeat data_shot during meta-training.
train_meta.py, line.158
logits = model((data_shot.unsqueeze(0).repeat(num_gpu, 1, 1, 1, 1), data_query))

Thanks for your nice work. I am new to FSL, so I am a little confused about supporting and query sets. In my understanding, we can only fine-tune the model on supporting sets, but I find you also use query set(each class 15), then what’s the difference between the case and 5way 20shot? I am really desperate for the answer. Thanks.

Thanks so much for providing the code of your paper.
I am new to FSL and trying to understand the training and data loader process. I understand the concept of support and query as well as shot and way. But I'm still confused about how you make the data loader and appreciate if you can explain it. I also read the paper several times but it didn't work to understand these details. I really appreciate your help in advance.

1. In the case of 1-shot 5-way, what does query=15 mean? how should we choose the number for query? does it need to be some specific number in agreement with previous works or it is arbitrary and kinda like hyperparameter?

in the cub dataset, we have 5892 images. and I see the data loader generate batch of 80x3x84x84 for input and the output is in forms of 80x640x5x5:

2. I don't understand where the 80 comes from at the first point, and then in the output where the 640 comes from?

3. later I see we have data_shot and logits, the size of logit is 75x5, can you please explain what 75 represent? I understand that it is 15*5=75 but I could not understand what it represents.

Thanks in advance for your help.

Does it possible to apply DeepEMD to general classification task?

Hi,

Thanks for your inspring work! However, I have a question.

I notice that in your readme file of the project, a command is provided to meta-train the model, after pre-training is done, where opencv is specified as the solver:

#use opencv solver (about 8GB memory)
$ python train_meta.py -deepemd fcn -shot 1 -way 5 -solver opencv -gpu 0,1,2,3

Nevertheless, in your paper, 5.1 Implementation Details, Training paragraph, it's claimed that QPTH is used during meta-training: “At training time, we use the GPU accelerated convex optimization solver QPTH [76] to solve the Linear Programming problem in our network and compute gradients for back-propagation.”

So, my question is:

1. Is opencv available during meta-training?
2. If both opencv and QPTH are available during meta-training? Does that mean for opencv, additional coding work has been done in the project to calculate gradients for back-propagation, like what has been done in QPTH?

Best Regards,
Neil

I have tried FCN pyramid structure 5,2,1 in the paper, but the accuracy on miniImagenenet is still 65.5%.
Besides, acc on tiered imagenet is 70.6%(2% lower), 72.8%(2% lower) on CIFAR-FS, 45.7% on FC100(1% lower) and 76.6%(0.5% lower).
I used raw code and keep the pyramid FCN as 5,2,1, all the results are lower than paper.

Hello, do you know how to generate the pickle file for miniImagenet?

Hi, this is very good work.
After running the code, I find a problem of the meat-train.
I use the CUB dataset. With the number of epochs increasing during pre-train, the validation accuracy on 5-way 5-shot task also increases. This is normal.
However, during the meta-train, where the feature extractor also gets updated, I can't find accuracy improvement or meta-train loss decreasing. The test accuracy after meta-train is similar to the validation accuracy after pre-train. Is that normal?
So, my question is, does the meta-train take effect?
Thanks.

I use the script and the pre-trained model provided by the auther (resnet12), only get 70.5% on 1-shot tiered-imagenet testset, which is at least 4% behind what's reported in the paper. Can other people reproduce the results on tiered?

What are the best hyperparameters for pre-training on TieredImageNet?

Should I keep the hyperparameters in the train_pretrain.py unchanged？

I notice that the CUB dataset you use is the accurately cropped version, not the same as in "Take a closer look at FSL". Is this somehow unfair?

So i'm able to train my own model on a custom dataset, but had a hard time figuring own how to do inference on this. For example given a single image, how do I make it so that the result is the prediction of what class the image belong to and the confidence scores of the prediction .

I downloaded the tiered imagenet dataset using the google drive link you provided but found that those images are not actually the same as the original dataset, which is usually the one people are using for fair comparison i think.

The original dataset contains images that have been pre-processed into 84*84, but in your dataset, seems like the images are all in 224*224. Could you please:

1. release the details about how to do the data pre-processing?
2. give some reference that any baseline methods are using the same dataset format?

Thanks!

In file Network.py line 110:

       for i in range(num_query):
                 for j in range(num_proto):



                _, flow = emd_inference_opencv(1 - similarity_map[i, j, :, :], weight_1[i, j, :], weight_2[j, i, :])

                similarity_map[i, j, :, :] =(similarity_map[i, j, :, :])*torch.from_numpy(flow).cuda()

        temperature=(self.args.temperature/num_node)

        logitis = similarity_map.sum(-1).sum(-1) *  temperature

The calculation of the flow, i.e., the best match between features, is conducted when forward logits and losses, but does the back-propagation go through the 'emd_inference_opencv' module? If I understand correctly, this module is run on cpu() and numpy, therefore no gradient is tracked.

Hello, first of all, thank you very much for your excellent work！！ When I run meta-training (meta-val), I found a problem, I would like to ask you.
I ran the experiment on 12 GB NVIDIA Titan V GPUs. I found that during meta-training ( meta-val ), the GPU utilization has been fluctuating and at a relatively low level. I don't know if you have found the same problem, or if it was my fault. If you have found this phenomenon as well, is it a bug, or is it inevitable?
This situation does not exist in the pre-training phase and only exists when meta-training or meta-validation, meta-testing is involved.

Looking forward to your reply. Thank you！！

Hi，I use the parameters you gave to train on miniimagenet，the model is DeepEMD-FCN. But the acc_test=60.5072. Did you use another training strategy, or did you pre-train on ImageNet？Looking forward to your reply

今天我刚刚读过您（们）的文章，确实是很有启发性的思路（尤其是structured FC相关的讨论）！然而，文章并没有详细解释训练的细节，同时我注意到文章中使用的多个对比方法的结果都相当高（这些应该是您用ResNet-12作backbone自行复现的结果），故而我对您训练/推理时的具体流程产生了兴趣，能请您分享一些相关细节吗？谢谢！
关于“结果相当高的对比方法”：举例来说，表4a中使用ResNet-12作backbone的ProtoNe在 miniImagenet 5way 5shot上有78.01%，而根据我之前的实验结果，在不使用数据增强、并使用了预训练backbone的情况下训练20万个episode后，这个数值在71%左右；CTM中也提及了，用ResNet-18作backbone的ProtoNet在5way 5shot上的准确率在74%。

I've just read your paper and I have to say that it's really thoughtful and inspiring ( especially the discussions on the structured fully connected layer). However, the details on the training procedure aren't discussed in the paper; and I notice that the results of many previous state-of-the-arts (which are obtained by your reproduction, I suppose) are relatively high in the paper than what I used to know, which makes me wonder how you managed to get them. Could you please reveal more details on your training/inference procedure? Thanks a lot.

About "relatively high in the paper": For example, in Table 4(a) ProtoNet(with ResNet-12 as its backbone) gets an accuracy of 78.01% on miniImagenet 5-way 5-shot tasks. However, according to my previous experiments, the performance of ProtoNet+ResNet12 is around 71% after 200,000 episodes of training, when a pretrained backbone is adopted and data augmentation is not used. It's also mentioned in CTM that ProtoNet+ResNet18 shows a performance of ~74% on miniImagenet 5-way 5-shot tasks.

Hello, thank you very much for sharing the source code.
I have a question and don't know how to solve it.
How does the trained model perform inference to achieve classification?

can you please tell me how should i install qpth?

Hi, I'm trying to pre train with grid or sampling mode but I have an error.

This is due to encode function when using patches: the outputs features are size (batch_size, feature_size=640, number_patchs) which can not be fed into Linear layer. How do you resize the outputs features of patches to feed properly into the linear layer ?

The complete error:

Traceback (most recent call last):
  File "train_pretrain.py", line 119, in <module>
    logits = model(data)
  File "/home/ubuntu/yohann/DeepEMD/deepemdenv/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/ubuntu/yohann/DeepEMD/deepemdenv/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 159, in forward
    return self.module(*inputs[0], **kwargs[0])
  File "/home/ubuntu/yohann/DeepEMD/deepemdenv/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/ubuntu/yohann/DeepEMD/src/models/deep_emd.py", line 29, in forward
    return self.pre_train_forward(input)
  File "/home/ubuntu/yohann/DeepEMD/src/models/deep_emd.py", line 41, in pre_train_forward
    return self.fc(self.encode(input, dense=False).squeeze(-1).squeeze(-1))
  File "/home/ubuntu/yohann/DeepEMD/deepemdenv/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/ubuntu/yohann/DeepEMD/deepemdenv/lib/python3.6/site-packages/torch/nn/modules/linear.py", line 93, in forward
    return F.linear(input, self.weight, self.bias)
  File "/home/ubuntu/yohann/DeepEMD/deepemdenv/lib/python3.6/site-packages/torch/nn/functional.py", line 1692, in linear
    output = input.matmul(weight.t())
RuntimeError: mat1 dim 1 must match mat2 dim 0

Hi, thanks a lot for the great work!

But I am a little bit confused about the resolution of the input images and I don't see you explained it explicitly in the original paper. I took a look at your dataloader code and seems like they are taking the original-size image (as large as 300*300) and do randomresizecrop (into 84) during training? please correct me if my understanding is wrong.

If yes, I am quite concerned that the comparison is quite unfair then? Cause according to my knowledge, most numbers on mini-imagenet are using the pre-resized 84 as input (except closer look at few-shot and simpleshot)?

Thanks in advance and looking forward to ur reply!

Hi, thanks for your great work! How should I train the 5-shot model? Direct training from scratch or use 1-shot model as initialization? Do relevant parameters such as learning rate need to be adjusted accordingly?

I find that sfc_lr default value is 0.1 in train_meta.py. However, the default value in eval.py is 100. Is it based on some considerations or just a bug?

Hello, first of all thanks for your excellent work!
But when I try to reproduce the results that you provided, what I got is pretty low.

Here is the complete log for MiniImageNet DeepEMD-FCN 1-Shot 5-Way:

C:\Users\admin\Anaconda3\envs\tensorflow\python.exe D:/Projects/DeepEMD/eval.py
{'data_dir': 'your/default/dataset/dir',  # it's fine because I make some changes to the fcn\mini_imagenet.py and already hardcode the real path.
 'dataset': 'miniimagenet',
 'deepemd': 'fcn',
 'feature_pyramid': None,
 'gpu': '0', 
 'metric': 'cosine',
 'model_dir': 'D:\\Projects\\DeepEMD\\trained\\deepemd_trained_model\\miniimagenet\\fcn\\max_acc.pth',
 'norm': 'center',
 'num_patch': 9,
 'patch_list': [2, 3],
 'patch_ratio': 2,
 'query': 15,
 'seed': 1,
 'set': 'test',
 'sfc_bs': 4,
 'sfc_lr': 100,
 'sfc_update_step': 100,
 'sfc_wd': 0,
 'shot': 1,
 'solver': 'opencv',
 'temperature': 12.5,
 'test_episode': 5000,
 'way': 5}
manual seed: 1
use gpu: [0]
loading model from : D:\Projects\DeepEMD\trained\deepemd_trained_model\miniimagenet\fcn\max_acc.pth
batch 5000: This episode:49.33  average: 57.1491+0.2795: 100%|██████████| 5000/5000 [38:01<00:00,  2.19it/s]
test Acc 57.1491
Test Acc 57.1491 + 0.2795

Process finished with exit code 0

I think if you could publish the complete arguments it will help to resolve this problem.

Thx in advance!

Hello, thanks for your great work first.

I want to reproduce some results on 5-shot tasks but found the training process quite slow compared to 1-shot tasks (e.g. 10-20 seconds/iterations), is it normal? or could you share the training configs for 5-shot tasks.

btw, I also have questions about how you improve your numbers on miniimagenet 5-way 1-shot task from CVPR version (65.91) to arxiv version? any suggestion is appreciated.

Thanks
Yang

Hi thanks a lot for your previous detailed response and it's really helpful. However, I am still quite confused about the details in table 3c.

In the last post, u mentioned: "For the results in Table3C, part of some is based on our implementation, while for the others , we added the citation behind the method name, which you can refer to."

so in this table above, could u tell us in details:

1. which number is implemented by yourselves and which one is citing others' number?
2. for the numbers implemented by yourselves, they are using the same data input setting as FEAT and deepEMD, right? I mean the input image is pre-cropped by the given bounding box.

Thanks a lot!