xyupeng / contrastivecrop Goto Github PK

Thanks for released the brilliant work again! When i was in traing process, i found that if i resume the training, the process will rsume from the saved checkpoint, but the self.use_box will return to the False, i mean the type of crop will becam to random crop, i was confused whether that will bother the end of training? or should i set the warmup_epochs to zero, when i resume trainging, in order to make sure that the type of crop is same as previously? Thanks for your reply.

What does Bx0 and Bx1 stand for in pseudocode

If I want to use representation information for downstream tasks, which part of the feature should I use ？

In the ContrastiveCrop.py file:
Line 44 to 47:

ch0 = max(int(height * h0) - h//2, 0)
ch1 = min(int(height * h1) - h//2, height - h)
cw0 = max(int(width * w0) - w//2, 0)
cw1 = min(int(width * w1) - w//2, width - w)

the upper bounds ch1 and cw1 may be wrong?
It should be the addition instead of the subtraction? From my understanding, it ch1 and cw1 give the upper bounds (the maximum of the cropping region). Or I misunderstand the cropping candidate regions.

···
ch1 = min(int(height * h1) + h//2, height - h)
···
cw1 = min(int(width * w1) + w//2, width - w)

Thanks in advance.

Thanks for sharing your great job! when i finish the pretext task in my own dataset, i got a checkpoint file, i check the keys in that file, it conbines statedict and boxes, i was confused about the box information is useful in the downstream task? or should i also consider to load the box weight?

作者你好，tiny-imagenet和stl10的configs训练文件能上传一下嘛？拜托了谢谢！！！

Hi there， thanks for your brilliant work, well...i have a question that if that work support custom dataset? thansk for your work again.

You wrote that alpha was set to 0.6 for your method in sec.4.2, while I saw that alpha was set to 0.1 in the config you are given on the github in the folder "small". So I'm wondering which one is used for the results in paper?

ch0 = max(int(height * h0) - h//2, 0)
ch1 = min(int(height * h1) - h//2, height - h)
In your implementation, ch0 can be larger than ch1. For example, ch0 and ch1 are both near 1.0.
I think this situation may cause some errors.

hello, i've tried use your work on my dataset, and it made work, too.But when i try to enlarge my dataset, there is a problem occured: when the model train to 200 or 300epochs, the box update always failed, i try to figure out the reason, found that when the process going to update, the heat_map tensor is empty, i was confused how to solve this problem and does this occured by my dataset size? because when i use small dataset, this problem dosent happend.

在ContrastiveCrop(alpha=alpha, size=224, scale=(0.2, 1.0))里面没有设置ratio，请问这个参数怎么设置

Hi, Thanks for your amazing work.

I got the following error when i trying to run the linear classification experiment of SimCLR:

Command:

python DDP_linear.py configs/linear/cifar10_res18.py --load ./checkpoints/small/cifar10/simclr_alpha0.1_th0.1/last.pth

Error:

  File "/home/futong/jiarunliu/ContrastiveCrop/DDP_linear.py", line 276, in main_worker
    load_weights(cfg.load, model, optimizer, resume=False)
  File "/home/futong/jiarunliu/ContrastiveCrop/DDP_linear.py", line 107, in load_weights
    for k, v in state_dict.items():
UnboundLocalError: local variable 'state_dict' referenced before assignment

And I have checked the aviaible keys in SimCLR's checkpoint:

>>> ckpt = torch.load("checkpoints/small/cifar10/simclr_alpha0.1_th0.1/last.pth", map_location='cpu')
>>> ckpt.keys()
dict_keys(['optimizer_state', 'simclr_state', 'boxes', 'epoch'])

Maybe you should consider change the key here?

Thank you for an interesting paper and easy to understand.
Can I ask some questions?

1/ I still don't understand what is the class score you mentioned in section 3.4.
Can you explain more?
I checked in the code but I couldn't find it. Please correct me if I missed it.

2/ It's interesting that the learning rate for training the linear classifier is 10.
Do you have any findings on this? or it's a heuristic configuration?

3/ What is the red plot in Section 4.4. Ablation Studies / Semantic-aware Localization

We also make comparison with RandomCrop that does not use localization (i.e., k = 0), and ground
truth bounding boxes (the red plot).

Is it another experiment but has been removed in Fig 6.a ?

Thank you

你好，请问后续会更新模型的权重和数据集吗？
另外想请问一下大佬，关于对比学习，如何去评价模型embedding是否学的“够好”（表征能力）？好像一般是根据下游任务去评价的，但是有些任务很难去定义这样子的任务。例如业务场景是那种社媒的？

how to use this code，thanks

A question about h_min, h_max = torch.nonzero(h_filter).view(-1)[[0, -1]] / Hi # [h_min, h_max]; 0 <= h <= 1 This line of code generates an error :IndexError: index is out of bounds for dimension with size 0 .Have you ever met?

Hi author, comparative cropping is a very good idea. We used the last layer feature map or attention map in a similar project in moco v3 to make cropped rectangular borders according to a threshold of 0.1, but both had poor results. We updated the bounding box at 20 intervals and printed the information of the 300 epoch rectangular border to find h_min=0,w_min=0,h_max=0.9955,w_max=0.9955. This indicates that all pixel points in the produced heat map are larger than the threshold and therefore do not play the effect of comparative cropping. Please, are we supposed to enlarge the threshold? In addition, we observe that only the left and lower boundaries of the rectangular borders are related to the heat map, but the upper and right borders are uniformly sampled and computed according to the given scale and ratio. Then, if only the left and bottom borders of the rectangular border are considered, and the right and top borders are randomly selected can we guarantee that the main objects in the image are in the bounding box. This is also the point that makes us wonder. We hope you can give us an answer to the above question, thank you very much.

Hi, great work! Thank you for sharing.
In the codes, training data is loaded after the contrastive crop and then is used for training through all epochs.
And for every location interval epoch, the box is updated.
The box is used for the contrastive crop in the data loader process to generate better sample pairs.
So how is the updating of the box used for generating better sample pairs?
I guess maybe we should load the training data every time we update the box, what do you think?