kanezaki / pytorch-unsupervised-segmentation-tip Goto Github PK

Hi,
Thanks for your amazing work!!
I just wonder how to align the predicted label with the ground truth.
For example, given an input and fixed network, we could do the prediction (fixed) . We could use the argmax function as pseudo labels during training period. However, how could we calculate the mIoU metric with ground truth and predicted results.
as we could assign the pixel belonging to a dog to label "0" when annotating the data set, we could also assign it to label "1", it's not sure.

Thanks~

Hello,
I always get the following error when running the demo code multiple times even if I am using a very tiny image. I think maybe the cache is not emptied or something? I am not really sure and would appreciate your help. Thank you

Error Message:

Traceback (most recent call last):
File "demo.py", line 127, in
lhpy = loss_hpy(HPy,HPy_target)
File "/home/omar/anaconda3/envs/dlr/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/home/omar/anaconda3/envs/dlr/lib/python3.7/site-packages/torch/nn/modules/loss.py", line 88, in forward
return F.l1_loss(input, target, reduction=self.reduction)
File "/home/omar/anaconda3/envs/dlr/lib/python3.7/site-packages/torch/nn/functional.py", line 2191, in l1_loss
ret = torch._C._nn.l1_loss(expanded_input, expanded_target, _Reduction.get_enum(reduction))
RuntimeError: CUDA out of memory. Tried to allocate 60.00 MiB (GPU 0; 3.95 GiB total capacity; 708.35 MiB already allocated; 111.00 MiB free; 742.00 MiB reserved in total by PyTorch)

Hi, I have some questions concerning the evaluation and the dataset.

1. Is it possible to share the code to produce the mIOU score?
   Since this IOU is used when the number of clusters is different between GT and outputs, which is fundamentally different from mIOU in other applications. It would be nice if you can share the code so followers can use it without misunderstanding.

2. In Table 1, did you use all data in VOC, BSD datasets? or only the training set? or the validation set? or just a few of them displayed in your paper?

3. In Table 1, if my understanding is correct, the algorithm is performed on ALL images in these datasets one by one. How long does that take? Since one run takes several minutes, I suppose that would be a rather LONG time.

4. Is it possible to share the code to pre-process the VOC dataset? As you mentioned in your paper "the object category labels in PASCAL VOC 2012 dataset were ignored and each segment along with the background region was treated as an individual segment." I'm not sure what that means.

Thanks for your time. It would be nice if you can answer these questions.

Hi, can you share the code how to draw precision-recall curve?
Thank you so much

Hi I was wondering if it was possible to train on multiple images instead of just a single image? I have been able to apply the model to my own data but I'd like to train it on a full training set ideally.

When using scribble loss with the demo picture (cat) 2007_001774.jpg and 2007_001774_scribble.png in the loss function I get the error:

IndexError: Target 8 is out of bounds.

if args.scribble:
    loss = args.stepsize_sim * loss_fn(output[ inds_sim ], target[ inds_sim ]) + args.stepsize_scr * loss_fn_scr(output[ inds_scr ], target_scr[ inds_scr ].long()) + args.stepsize_con * (lhpy + lhpz)
else:
    loss = args.stepsize_sim * loss_fn(output, target) + args.stepsize_con * (lhpy + lhpz)

Traceback (most recent call last):
  File "C:\Users\PycharmProjects\scribbleLoss\main.py", line 175, in <module>
    stepszie_scr *loss_fn_scr(output[inds_scr], target_scr[inds_scr].long()) + \
  File "C:\Usersl\PycharmProjects\scribbleLoss\venv\lib\site-packages\torch\nn\modules\module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "C:\Users\PycharmProjects\scribbleLoss\venv\lib\site-packages\torch\nn\modules\loss.py", line 1120, in forward
    return F.cross_entropy(input, target, weight=self.weight,
  File "C:\Users\PycharmProjects\scribbleLoss\venv\lib\site-packages\torch\nn\functional.py", line 2824, in cross_entropy
    return torch._C._nn.cross_entropy_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index)
IndexError: Target 8 is out of bounds.

Without scribble everything is working fine and I do not see the issue. I also added the .long() from issue #4 but it seems like I still have another issue here.

Awesome work guys! :)

I'm looking at the code and I'm just wondering if there is any theoretical limitation to training the reference image with scribbles?

Thank you very much!

Firstly, I want to express my gratitude for your valuable work.
I have a quick question regarding the evaluation aspect of your code. Could you kindly specify which subset of the Pascal VOC 2012 dataset you used for model evaluation? Additionally, I am interested in understanding the methodology behind calculating the mean Intersection over Union (mIoU). Any insights you can provide would be greatly appreciated.

Thank you again for your time and contributions.

Hello!

First of all, thank you for your great code and papers.

There was a problem using the scribble option, so I was asked a question.

It works normally without the scribble option, but when it works with the scribble image, it does not work with the following error.

The scribble image I created myself is an 8-bit png file like the scribble image in the repository.
I tried referring to 4.B of the paper, but I couldn't find any other problems.

Any good opinions?

python demo.py --input ./BSD500/101027.jpg
/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/nn/_reduction.py:43: UserWarning: size_average and reduce args will be deprecated, please use reduction='mean' instead.
warnings.warn(warning.format(ret))
qt.qpa.xcb: could not connect to display
qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "/home/ubuntu/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/cv2/qt/plugins" even though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

Available platform plugins are: xcb, eglfs, minimal, minimalegl, offscreen, vnc, webgl.

Aborted (core dumped)

Thank you very much for your work, I think it is very simple and effective！
Scribble is a very interesting extension module, how did you get the scribble?

Hello I am trying to quantize, evaluate and convert this model into a xmodel using pytorch. I am curious if maybe you have done it, because I am having problems trying to quantize it using the resnet18_quant.py quantizer.

Hope someone can help me

Hi, thank for your nice work and job.
After reading your code, i have a little question. can you share the code how to compute the mIoU in unsupervised way, because which is different from supervised segmentation.
Thank you!👍

Traceback (most recent call last):
File "demo.py", line 145, in
loss = args.stepsize_sim * loss_fn(output[ inds_sim ], target[ inds_sim ]) + args.stepsize_scr * loss_fn_scr(output[inds_scr], target_scr[inds_scr])
File "d:\tools\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "d:\tools\Anaconda3\lib\site-packages\torch\nn\modules\loss.py", line 962, in forward
ignore_index=self.ignore_index, reduction=self.reduction)
File "d:\tools\Anaconda3\lib\site-packages\torch\nn\functional.py", line 2468, in cross_entropy
return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
File "d:\tools\Anaconda3\lib\site-packages\torch\nn\functional.py", line 2264, in nll_loss
ret = torch._C._nn.nll_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index)
RuntimeError: Expected object of device type cuda but got device type cpu for argument #2 'target' in call to _thnn_nll_loss_forward

Hi, great work! Could you explain more about how to test the work on the whole BSD500/VOC 2012 dataset? Thanks!

Hello,

I'm having an issue with the code. I'm using colab, having all the necessary requirements installed. I forked the project into my colab session then run the python command : "!python3 demo.py --input ./BSD500/101027.jpg"
to generate ouput. It instead prints the following in the std :
/usr/local/lib/python3.7/dist-packages/torch/nn/_reduction.py:42: UserWarning: size_average and reduce args will be deprecated, please use reduction='mean' instead.
warnings.warn(warning.format(ret))
: cannot connect to X server
The output images isn't generated

Hi, thank you for your sharing very much!
I have a question that why there are multiple P-R curves in the paper.

It seems that for every p-r curve plot, the IOU threshold is specific, then what another parameter or threshold is changed to influence the precision and recall rate in each plot?

A snippet from paper:

A snippet from the given code:

From the above figures I assume µ == (lhpy + lhpz). Am I right?