jfzhuang / davss Goto Github PK

Thanks for sharing so wonderful video semantic segmentation work.

I did the experiments on Cityscapes dataset, and I can reproduce the data in your paper. However, I have encountered some problems when training the Camvid dataset. (The pretrained models are downloaded from the links you provided, and the requirements are satisfied.)

The PDA curves on testing are very weird, but the cfnet loss and the semantic loss are decreasing. Attached are the IoU accuracy across different testing epochs when the propagation distance is 1, 5, and 9, respectively. The last two images are the training losses.

I am confused about the above performances. The model seems over-fitting. Have you seen this kind of problem in your training? Do you have any idea how to fix this training problem?
In the "train.sh" file, both experiments adopt the same training epochs, learning rates, adjustment schedule of learning rates. Is there anything wrong with this setting?

Besides, I can obtain the exact same segmentation performances on Camvid just like the images in the repository folder "result --> spatial_correction_camvid" when I loaded the trained "spatial_correction_camvid" model provided in this repository. But unfortunately, I cannot achieve the same PDA in your paper. I can't figure it out.

      Fig. 12 in your paper

      The PDA curve using the trained "spatial_correction_camvid" model provided in this repository.

Looking forward to your response. Many thanks.

LINK : fatal error LNK1181: 无法打开输入文件“C:\video semantic segmentation\DAVSS-master\lib\model\resample2d_package\build\temp.win-amd64-3.7\Release\resample2d_cuda.obj”
error: command 'D:\exe\visual studio\VC\Tools\MSVC\14.28.29910\bin\HostX86\x64\link.exe' failed with exit status 1181

当我训整个框架的时候，出现了：
THCudaCheck FAIL file=/pytorch/aten/src/THCUNN/generic/SpatialClassNLLCriterion.cu line=134 error=77 : an illegal memory access was encountered
Traceback (most recent call last):
File "exp/spatial_correction_cityscapes/python/train.py", line 292, in
train()
File "exp/spatial_correction_cityscapes/python/train.py", line 181, in train
loss_semantic, loss_cfnet = net(img_list, label=gt_label)
File "/data/anaconda3/envs/wang-davss/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "/data/anaconda3/envs/wang-davss/lib/python3.7/site-packages/torch/nn/parallel/distributed.py", line 445, in forward
output = self.module(*inputs[0], **kwargs[0])
File "/data/anaconda3/envs/wang-davss/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "/data/video_network/DAVSS/lib/model/scnet.py", line 75, in forward
loss_semantic += self.semantic_loss(img_3_out_propagate, label)
File "/data/anaconda3/envs/wang-davss/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "/data/anaconda3/envs/wang-davss/lib/python3.7/site-packages/torch/nn/modules/loss.py", line 932, in forward
ignore_index=self.ignore_index, reduction=self.reduction)
File "/data/anaconda3/envs/wang-davss/lib/python3.7/site-packages/torch/nn/functional.py", line 2317, in cross_entropy
return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
File "/data/anaconda3/envs/wang-davss/lib/python3.7/site-packages/torch/nn/functional.py", line 2117, in nll_loss
ret = torch._C._nn.nll_loss2d(input, target, weight, _Reduction.get_enum(reduction), ignore_index)
RuntimeError: cuda runtime error (77) : an illegal memory access was encountered at /pytorch/aten/src/THCUNN/generic/SpatialClassNLLCriterion.cu:134
terminate called after throwing an instance of 'c10::Error'
what(): CUDA error: an illegal memory access was encountered (insert_events at /pytorch/c10/cuda/CUDACachingAllocator.cpp:771)

然而不知道出了什么问题，不知道作者遇到过没有，希望予以解答。

I like to test your pretrained cityscapes model with my own video clip.
I can extract each frame from my own video, but I don't see you have a script to do inference of test images from a video.
Can you provide a guidance of how to do this?

Thanks,