microsoft / transductive-vos.pytorch Goto Github PK

Thanks for sharing your excellent code!
Do you have plan to release the training and inference code on Youtube-VOS dataset?

Oh, someone has asked the same question. Sorry to bother.

Hi，Thanks for your excellent work.

I encounter a problem when I follow your indication "python -m torch.distributed.launch --master_port 12347 --nproc_per_node=4 main.py --data /path-to-your-davis-directory/" during training.

Here is my problem:

• AssertionError: DistributedDataParallel device_ids and output_device arguments only work with single-device CUDA modules, but got device_ids [3], output_device None, and module parameters {device(type='cuda', index=3), device(type='cpu')}.

My device information is showed below:

• GeForce RTX 2080 Ti
• Ubuntu 18.04
• pytorch 1.5.0 & torchvision 0.6.0
• 4 GPUs with 11GB Memory

How should I solve this problem?
Thank you again.
Best wishes.

Hello,
During the loss calculation process, the temperature is set to 1. What does it mean?
Thank you!

Hi, Thank you for producing this repository. It is very impressive that you are able to achieve these results with such small sample size.
I'd like to train a similar model on my dataset, which includes only three classes, but the object I want to detect is very difficult to see (it's very small and moves quickly and sporadically). I need to use multiple annotated videos for each class, but in the Davis dataset there is only one video for each class. How could I use multiple videos for each class instead?
Also, since the object I'm looking for is very small and hard to see, could I provide some videos where there is no object at all (the segmentation mask would be blank). I'm not clear on how I would do this given the current configuration.

I've downloaded the davis_train.tar and youtube_train.tar, but I can't successful unzip the tar package, it seems that some data is corrupted, I don't know where the problems come from...

This repository is currently missing a LICENSE file.

A license helps users understand how to use your project in a compliant manner. You can find the standard MIT license Microsoft uses at: https://github.com/microsoft/repo-templates/blob/main/shared/LICENSE.

If you would like to learn more about open source licenses, please visit the document at https://aka.ms/license (Microsoft-internal guidance).

How to inference under the input of high-resolution images (1080p), I often have memory overflow problems, do you have any good ideas? Thank you!

Hi,

I've trained and did inference using the given commands in this repo with 4 GPUs. I trained on DAVIS17 training set and evaluated using the DAVIS17 validation set. I used this repo https://github.com/davisvideochallenge/davis2017-evaluation for evaluation.

but I was only able to get the following result:

--------------------------- Global results for val ---------------------------                                                                                                                                     
 J&F-Mean    J-Mean  J-Recall   J-Decay    F-Mean  F-Recall   F-Decay                                                                                                                                              
  0.66228  0.642125  0.758698  0.198392  0.682435  0.807384  0.242924   

I trained for 240 epochs as mentioned and evaluated on the 240th epoch. I also evaluated a few checkpoints around 240th but the result was very similar.

Could you please help me point out what could be the issues?

How to use one GPU for training? How to adjust the parameters?

Do you have any plan to release the code for youtubevos training and evalutation?

What is the cuda version of your project？
This is my error：
RuntimeError: cuda runtime error (35) : CUDA driver version is insufficient for CUDA runtime version at torch/csrc/cuda/Module.cpp:32

I use two gpus, how do I modify the parameters?thank you

Hello！In the log.txt file during training, the time on the training set and validation set is displayed. And how is the fps obtained? Is time divided by 25? Thank you very much!

Thanks for sharing your codes. I tried to train a model on DAVIS2017 dataset, but I found there's no file named "annotation_centroids.npy", which should be loaded here. Do you plan to share this file and what's the meaning of it?

The previous discussion is here: #19

I rerun with a smaller learning rate, and use the latest code. However the results are still not being matched.

Per-sequence results saved in outputs//per-sequence_results-DAVIS17val.csv
--------------------------- Global results for DAVIS17val ---------------------------
 J&F-Mean    J-Mean  J-Recall   J-Decay    F-Mean  F-Recall   F-Decay
 0.663763  0.644109  0.754703  0.202716  0.683417   0.80778  0.248114

The paper reports 72 J&F-Mean.

If anyone reproduces it, please leave your results as a reference. And I am very curious that the Sync-BN can boost 6 points in total? Thanks in advance.

By the way, I am not here for provocation's sake, instead it is a problem worthing to be discussed. I will not use TVOS as my baseline.

Hi, Thank you for your excellent work!

I want to know what the 'annotation_centroids.npy' and the centroids means.
To be specific，I can't find something in paper related to torch.argmin(torch.sqrt(torch.sum((img.unsqueeze(1) - centroids) ** 2, 2)), 1) in function 'rgb2class'.

Best wishes.

Here is my question in Chinese，which is consistent with the above:

'centroids'这个变量以及在dataset中的 'annotation_centroids.npy' 发挥了什么作用呢，我在论文中没有看到对应的信息。
关于'rgb2class'函数中的 torch.argmin(torch.sqrt(torch.sum((img.unsqueeze(1) - centroids) ** 2, 2)), 1) 具体含义麻烦您能解释一下吗

祝好。

Thanks for sharing such a neat implementation

When I try to understand the effect of the motion prior, I ran inference without the motion prior by commenting off the following lines:

However, the final score was not the same as what was reported in the paper (J=69.0 in Table 2). Here is what I got:
+--------------+--------+----------+---------+--------+----------+---------+
| Method | J_mean | J_recall | J_decay | F_mean | F_recall | F_decay |
+--------------+--------+----------+---------+--------+----------+---------+
| val-no-motion | 0.618 | 0.714 | 0.224 | 0.675 | 0.772 | 0.253 |
+--------------+--------+----------+---------+--------+----------+---------+

Could you please help me understand what went wrong? What was the setting for the ablation study in the paper? Thank you!

Hi, thanks for releasing the codes and I run your code completely but the results cannot be reproduced at all.

I use DAVIS 2017 as my training set, and evaluate the checkpoints on DAVIS 2017 validation set.

However the results are:

G/J/F: 67.2/65.2/69.2

But in your paper:

G/J/F: 72.3/69.9/74.7

I notice you use 4 GPUS with 16GB memory each and here I only has 4 GPUs with 11GB memory each. I think the hardware difference should NOT make such a significant difference. Could you please have an explanation on that because there are a few fellows issuing the same problems below.

Thanks in advance.

And what's more, your paper says it is a transductive method, however, your codes are TOTALLY different with the equations in your paper, and the masks are not semi-supervised, it is fully supervised by cross-entropy loss.

Please explain the issue which I think it is an essential problem in your paper.