bcv-uniandes / dms Goto Github PK

The download link to all pretrained weights seem to be down, for example this one.

Would it be possible to upload the pretrained weights on another platform?

Traceback (most recent call last):
File "E:/DMS-master/dmn_pytorch/train.py", line 179, in
max_query_len=args.time)
File "E:\DMS-master\dmn_pytorch\referit_loader.py", line 91, in init
self.corpus = torch.load(corpus_path)
File "D:\Anaconda\lib\site-packages\torch\serialization.py", line 525, in load
with _open_file_like(f, 'rb') as opened_file:
File "D:\Anaconda\lib\site-packages\torch\serialization.py", line 212, in _open_file_like
return _open_file(name_or_buffer, mode)
File "D:\Anaconda\lib\site-packages\torch\serialization.py", line 193, in init
super(_open_file, self).init(open(name, mode))
FileNotFoundError: [Errno 2] No such file or directory: 'data\unc\corpus.pth'

Traceback (most recent call last):
File "refer.py", line 39, in
from external import mask
File "/home/wam/refer-master/external/mask.py", line 3, in
import external._mask as _mask
ImportError: No module named _mask
Could you give me some advice to solve the error?

when I run the train.py in pycharm
/home/lanxiao/anaconda3/envs/torch/bin/python /home/lanxiao/DMS-master/dmn_pytorch/train.py
Argument list to program
--data ../referit_data
--split_root data
--save_folder weights/
--snapshot weights/qseg_weights.pth
--num_workers 2
--dataset unc
--split train
--val None
--eval_first False
--workers 4
--no_cuda False
--log_interval 200
--backup_iters 10000
--batch_size 1
--epochs 40
--lr 1e-05
--patience 2
--seed 1111
--iou_loss False
--start_epoch 1
--optim_snapshot weights/qsegnet_optim.pth
--accum_iters 100
--pin_memory False
--size 512
--time -1
--emb_size 1000
--hid_size 1000
--vis_size 2688
--num_filters 10
--mixed_size 1000
--hid_mixed_size 1005
--lang_layers 3
--mixed_layers 3
--backend dpn92
--mix_we False
--lstm False
--high_res False
--upsamp_mode bilinear
--upsamp_size 3
--upsamp_amplification 32
--dmn_freeze False
--visdom None
--env DMN-train
Train begins...
/home/lanxiao/anaconda3/envs/torch/lib/python3.6/site-packages/torch/nn/functional.py:2351: UserWarning: nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.
warnings.warn("nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.")
/home/lanxiao/anaconda3/envs/torch/lib/python3.6/site-packages/torch/nn/functional.py:2423: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
"See the documentation of nn.Upsample for details.".format(mode))
/home/lanxiao/anaconda3/envs/torch/lib/python3.6/site-packages/torch/nn/functional.py:2351: UserWarning: nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.
warnings.warn("nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.")
/home/lanxiao/anaconda3/envs/torch/lib/python3.6/site-packages/torch/nn/functional.py:2423: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
"See the documentation of nn.Upsample for details.".format(mode))
/home/lanxiao/anaconda3/envs/torch/lib/python3.6/site-packages/torch/nn/functional.py:2351: UserWarning: nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.
warnings.warn("nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.")
/home/lanxiao/anaconda3/envs/torch/lib/python3.6/site-packages/torch/nn/functional.py:2423: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
"See the documentation of nn.Upsample for details.".format(mode))
/home/lanxiao/anaconda3/envs/torch/lib/python3.6/site-packages/torch/nn/functional.py:2351: UserWarning: nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.
warnings.warn("nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.")
/home/lanxiao/anaconda3/envs/torch/lib/python3.6/site-packages/torch/nn/functional.py:2423: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
"See the documentation of nn.Upsample for details.".format(mode))
Traceback (most recent call last):
File "/home/lanxiao/DMS-master/dmn_pytorch/train.py", line 445, in
train_loss = train(epoch)
File "/home/lanxiao/DMS-master/dmn_pytorch/train.py", line 275, in train
out_masks = net(imgs, words)
File "/home/lanxiao/anaconda3/envs/torch/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/home/lanxiao/DMS-master/dmn_pytorch/models/dmn.py", line 450, in forward
out, features = self.langvis(vis, lang)
File "/home/lanxiao/anaconda3/envs/torch/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/home/lanxiao/DMS-master/dmn_pytorch/models/dmn.py", line 156, in forward
lang, _ = self.lang_model(lang)
File "/home/lanxiao/anaconda3/envs/torch/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/home/lanxiao/anaconda3/envs/torch/lib/python3.6/site-packages/sru/cuda_functional.py", line 635, in forward
h, c = rnn(prevx, c0[i])
File "/home/lanxiao/anaconda3/envs/torch/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/home/lanxiao/anaconda3/envs/torch/lib/python3.6/site-packages/sru/cuda_functional.py", line 575, in forward
SRU_Compute = SRU_Compute_GPU(self.activation_type, n_out, self.bidirectional)
File "/home/lanxiao/anaconda3/envs/torch/lib/python3.6/site-packages/sru/cuda_functional.py", line 333, in init
self.compile()
File "/home/lanxiao/anaconda3/envs/torch/lib/python3.6/site-packages/sru/cuda_functional.py", line 348, in compile
prog = Program(SRU_CODE.encode(), 'sru_prog.cu'.encode())
File "/home/lanxiao/anaconda3/envs/torch/lib/python3.6/site-packages/pynvrtc/compiler.py", line 52, in init
include_names)
File "/home/lanxiao/anaconda3/envs/torch/lib/python3.6/site-packages/pynvrtc/interface.py", line 200, in nvrtcCreateProgram
c_char_p(encode_str(src)), c_char_p(encode_str(name)),
File "/home/lanxiao/anaconda3/envs/torch/lib/python3.6/site-packages/pynvrtc/interface.py", line 54, in encode_str
return s.encode("utf-8")
AttributeError: 'bytes' object has no attribute 'encode'
Exception ignored in: <bound method Program.del of <pynvrtc.compiler.Program object at 0x7f99511ed5f8>>
Traceback (most recent call last):
File "/home/lanxiao/anaconda3/envs/torch/lib/python3.6/site-packages/pynvrtc/compiler.py", line 56, in del
self._interface.nvrtcDestroyProgram(self._program)
AttributeError: 'Program' object has no attribute '_program'

what is the main question?

Establecer posibles puntos de falla del modelo original.

I have tried to reproduce the code. The code ran smoothly but the result was far below the ones as you mentioned in the paper, I used the weights you provide and max mIoU was 0.11. When I visualize it, it's totally messy. I was wondering the reason.

Necesitamos realizar las siguientes tareas:

• Implementar SoA
• Revisar las fallas del ViLSTM → Emilio (@milongo) (Ver #3)
• Implementar nuevo modelo

I have these words in readme in order to install refer
sudo pip install git+https://github.com/andfoy/refer.git
sudo pip install -U git+https://github.com/taolei87/sru.git@43c85ed --no-deps
But I still meet:
Traceback (most recent call last):
File "/home/lanxiao/DMS-master/dmn_pytorch/train.py", line 23, in
from dmn_pytorch.models import DMN
File "/home/lanxiao/DMS-master/dmn_pytorch/init.py", line 12, in
from .referit_loader import ReferDataset
File "/home/lanxiao/DMS-master/dmn_pytorch/referit_loader.py", line 21, in
from referit import REFER
ImportError: cannot import name 'REFER'

Hi,

I am attempting to use the DMN model with the pre-trained weights provided. However, whenever I evaluate the model using the below-mentioned command for UNC dataset using the pre-trained weights downloaded from the link provided in the ReadMe, I get a very low mIoU(close to 0.11).

Pre-trained weights( for unc dataset)

Command Used for evaluating the model

python -u -m dmn_pytorch.train --data referit_data --dataset unc --val testA --backend dpn92 --num-filters 10 --lang-layers 3 --mix-we --snapshot ./snapshots/dmn_unc_weights.pth --epochs 0 --eval-first --high-res

Result of Evaluation(using above command)

Evaluation done. Elapsed time: 343.907 (s) 
Maximum IoU: 0.1159133240581 - Threshold: 0.0000000000000

I have tried with other datasets also but I am unable to get anywhere close to the performance numbers mentioned in the ReadMe using the pre-trained weights. I am not sure what the issue is here. Below I provide the different versions of packages/libraries I am using.

Dependency Versions

Python 3.7
Pytorch 1.0
SRU 2.1.2 (latest version)
Cuda 9.0

Please note that I am getting low numbers for all datasets and have only provided UNC dataset(and numbers) as an example.

Hi,

I just read your paper and appreciate your well-organized code very much. But I cannot produce results using the pre-trained model.

I am not quite sure where the problem is. Did I use the wrong command?

python -u -m dmn_pytorch.train 
--data /home/xxx/dms_data 
--dataset unc 
--val testA 
--backend dpn92 
--num-filters 10 
--lang-layers 3 
--mix-we 
--save-folder ./checkpoints 
--snapshot /home/xxx/DMS/checkpoints/dmn_unc_weights.pth 
--epochs 0 
--eval-first

1. How much impact it will have on the mIoU when I adjust size from 512 to 384 or 256 with --size. I got quite lower result compared with your result. The mIoU of the referit val down to 0.47 and unc to 0.36 when I use --size. I am wondering whether it is reasonable result.

2. I am thinking of using bounding box rather than mask to locate the object with language guide. How could I use your code to train and test that? Is there any suggestions or instructions.

Appreciated for your efforts to reply

Hi, I try to run this code, but get the following error. where should I download this missed 'data/unc/corpus.pth' file ?

Traceback (most recent call last):
File "/usr/local/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/usr/local/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/media/wangxiao/b8efbc67-7ea5-476d-9631-70da75f84e2d/reference_code/DMS/dmn_pytorch/train.py", line 177, in
max_query_len=args.time)
File "/media/wangxiao/b8efbc67-7ea5-476d-9631-70da75f84e2d/reference_code/DMS/dmn_pytorch/referit_loader.py", line 89, in init
self.corpus = torch.load(corpus_path)
File "/usr/local/lib/python3.6/site-packages/torch/serialization.py", line 356, in load
f = open(f, 'rb')
FileNotFoundError: [Errno 2] No such file or directory: 'data/unc/corpus.pth'

Can you guide me test on my own dataset? How data needs to be created and things to keep in mind?

Hi, i notice that the network is set to train mode in evaluate function. Is that a special setting? Does this line affect the results of evaluation?

Sorry, I cannot obtain your data. There is an error.
ModuleNotFoundError: No module named 'referit.external._mask'

Hi, I'd like to fine tune my model weights using my own dataset based on your pretrained weights, like dmn_referit_weights.pth, I don't know how to do that because I have no optimizer weights to resume the training. Any instruction or suggestion will be appreciated for that.

I reproduced your model and got the same result on UNC dataset / val split. However, when I try to visualize the output, the segmentation mask is messy. Could you let me know how to visualize the output like as this figure?

when i train the project on the GPU0(8G)
i get:
RuntimeError: CUDA out of memory. Tried to allocate 9.50 MiB (GPU 0; 7.93 GiB total capacity; 6.49 GiB already allocated; 14.81 MiB free; 30.49 MiB cached)

when i train the project on the GPU0(12G)
i get:
RuntimeError: CUDA out of memory. Tried to allocate 16.88 MiB (GPU 0; 11.90 GiB total capacity; 10.58 GiB already allocated; 18.44 MiB free; 62.29 MiB cached)

when i train the project on the GPU0(12G) GPU1(12G)
I add:
if args.cuda:
net = nn.DataParallel(net , device_ids=[0,1])
net.cuda()
i get:
RuntimeError: CUDA out of memory. Tried to allocate 16.88 MiB (GPU 0; 11.90 GiB total capacity; 10.58 GiB already allocated; 18.44 MiB free; 62.29 MiB cached)

what can I do to solove this question?Thanks

According to link of download_data.sh . I can't download referit_splits.tar.bz2，can you offer it to me ？thanks a lot. I hope you can link to the network disk or send it to my email. Thank you again. email:[email protected]