fuy34 / superpixel_fcn Goto Github PK
View Code? Open in Web Editor NEW[CVPR‘20] SpixelFCN: Superpixel Segmentation with Fully Convolutional Network
License: Other
[CVPR‘20] SpixelFCN: Superpixel Segmentation with Fully Convolutional Network
License: Other
Thanks for your excellent work, I want if there any code support Python3? Thank you.
Hi, thanks for open-sourcing this awesome work.
If I understand correctly, the number of superpixels are controlled by the size of the input image (Section 3.4 in the paper). In my case, I would like to get relatively large superpixels, say, only 50 superpixels per image. I set downsize=64
in the run_demo.py
file and b_enforce_connect=False
, but it produces an image like this:
Also in Fig. 5, the model produces more than 1000 superpixels, is this done by inputting an image of size 512*512?
Any help would be appreciated!
Hello, I would like to ask how many epochs you have trained to achieve this effect?
Hello,
thanks first of all for providing the code to a very interesting paper. My question is about superpixel based downsampling. Here the association map and a LABXY_feat_tensor (consisting of property and location vector) is used. Does that downsampling work without using the property vector? Could you please tell me how this was implemented in the application with PSM net?
Hello author, I have two questions to ask.
Firstly, if I still use the BSDS500 dataset, I can set the superpixel size to 20 × 20 or 21 × 21 or any other value?
Secondly, how can I modify the code to achieve this? Directly modifying the downsize as shown in the figure may result in an error. I made some changes to the code myself, but the result was not ideal. I think I overlooked some important parameters. Thank you.
Our team is trying to merge SpixelNet with our task network as well to improve the segmentation boundary accuracy. We do believe this will greatly push the performance of the task network further! However, as we followed your supplement material, we have some confusions:
It will be really helpful if we can get some input from you directly! Thanks! : )
I would like to try Superpixel FCN, but I could not construct the environment to execute.
I used nvidia-docker to build the environment of python2.7, Pytorch0.4.1, CUDA9, and Ubuntu16.04. However, I don't know what python packages I should install.
If it's not too much trouble, can you tell me your environment of python2.7 and installed packages?
Shown in the following my Dockerfile.
FROM nvidia/cuda:9.1-cudnn7-devel-ubuntu16.04
USER root
RUN apt-get update && apt-get install -yq --no-install-recommends \
git \
python-dev \
python-pip \
python-setuptools \
&& apt-get clean && \
rm -rf /var/lib/apt/lists/*
RUN pip install torch==0.4.1 \
cython \
numpy==1.16.6 \
pillow==4.3.0 \
PyWavelets==1.0.3 \
networkx==2.2 \
matplotlib==2.1.2 \
scipy==1.2.3 \
scikit-image==0.14.1
RUN git clone https://github.com/fuy34/superpixel_fcn.git
RUN cd ./superpixel_fcn/third_party/cython/ && python setup.py install --user
When I run run_demo.py, it generates the unique maps as csv file, what does this file represent?
Hello, may I ask how to adjust the number of super pixels?
I have tried to install the requirements via conda, so it will check the requirements' compatibilities.
I get this error, which I searched and I found out, it's due to CUDA error. The new GPUs don't work with CUDA 9, and it's required to either compile torch 0.4.1 from source with CUDA 10, or install higher version of torch, which doesn't work with your code. Also, you didn't mention the torchvision version, but luckily conda would find the proper version(0.2.1). Could you please update your code for python 3+ and recent torch version?:
Traceback (most recent call last):
File "run_demo.py", line 169, in <module>
main()
File "run_demo.py", line 164, in main
time = test(args, model, tst_lst, save_path, n)
File "/home/yousef/anaconda3/envs/superpix_py36/lib/python3.6/site-packages/torch/autograd/grad_mode.py", line 46, in decorate_no_grad
return func(*args, **kwargs)
File "run_demo.py", line 97, in test
output = model(img1.cuda().unsqueeze(0))
File "/home/yousef/anaconda3/envs/superpix_py36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in __call__
result = self.forward(*input, **kwargs)
File "/home/yousef/Desktop/SpixelFCN/python3-version/superpixel_fcn/models/Spixel_single_layer.py", line 65, in forward
out1 = self.conv0b(self.conv0a(x)) #5*5
File "/home/yousef/anaconda3/envs/superpix_py36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in __call__
result = self.forward(*input, **kwargs)
File "/home/yousef/anaconda3/envs/superpix_py36/lib/python3.6/site-packages/torch/nn/modules/container.py", line 91, in forward
input = module(input)
File "/home/yousef/anaconda3/envs/superpix_py36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in __call__
result = self.forward(*input, **kwargs)
File "/home/yousef/anaconda3/envs/superpix_py36/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 301, in forward
self.padding, self.dilation, self.groups)
RuntimeError: cuda runtime error (11) : invalid argument at /opt/conda/conda-bld/pytorch_1535491974311/work/aten/src/THC/THCGeneral.cpp:663```
Hi
I find the image maybe is trained in rgb, not for lab? it's sure?
could you tell me which should i take, rgb or lab, to train?
Can I control all images to output the same number of superpixels?(Those photos have the same resolution.)
According to the traditional algorithm, such as SLIC, I can get a same number of superpixels in some photos.
And, further, can we control a certain number of pixels in a superpixel?
I not sure it we can do it with spixel_fcn?
Thank u very much.
Hello, I have a question here when run demo in my machine, can you help me please?
NameError: name 'enforce_connectivity' is not defined
File "/home/yangfq/PycharmProjects/pythonProject_sfcn/superpixel_fcn/train_util.py", line 243, in get_spixel_image
spix_index_np = enforce_connectivity(spix_index_np[None, :, :], min_size, max_size)[0]
NameError: name 'enforce_connectivity' is not defined
I have already try different version of connectivity library.
How can I solve this problem, Thank you very much.
Hi, Thanks for open-sourcing this awesome work. I noticed that this repo is for the segmentation and depth estimation tasks. Could you also provide the code or hyper-parameters for Eq.(4) in the paper? Thanks!
When I change the Variable "downsize" to 32, it becomes quite hard for the model to training, that is, the loss reduces slowly (have already trained 800 epochs).
Therefore, can I chose my own superpixels? Is there any special hyparameter that should be changed accordingly?
Dear authors,
The paper and the code are impressive. I have a simple question: In file "train_util.py", the definition of the variable "prob" used in functions "poolfeat()" and "upfeat()" is quite confusing. Concretely, for the first dim of "prob" (prob[:, 0, :, :]) in function "poolfeat()", it is used accompanied with the bottom right superpixel. However, in function "upfeat", this dim is used with the top left superpixel. I assume that in the paper, the superpixel and the prob in equation 1 and 2 should be consistent. What is the definition of every dim in the variable "prob" in your code settings?
Thanks!
When I evaluate my predicted results, I got a error as follows,, I don't know the reason. Hoping your response. Thank you very much.
ruby@ruby-MS-7D36:~/code/superpixel-benchmark/examples$ bash bash/my_eval.sh
54
bash/my_eval.sh: 行 43: /home/ruby/code/AINET/output/test_multiscale_enforce_connect/SPixelNet_nSpixel_54/map_csv: 是一个目录
the settings in the my_eval.sh as follows:
SUPERPIXELS=("54" "96" "150" "216" "294" "384" "486" "600" "726" "864" "1014" "1176" "1350" "1536" "1944") #
#SUPERPIXELS=("293" "1175") #
IMG_PATH=/home/ruby/code/AINET/test
GT_PATH=/home/ruby/code/AINET/test/map_csv
for SUPERPIXEL in "${SUPERPIXELS[@]}"
do
echo $SUPERPIXEL
/home/ruby/code/AINET/output/test_multiscale_enforce_connect/SPixelNet_nSpixel_${SUPERPIXEL}/map_csv $IMG_PATH $GT_PATH
done
the directory of ‘map_csv‘ as follows:
100007_img.csv
100039_img.csv
...
Why are there many isolated regions with identical IDs. The pixels assigned to identical superpixel is not together.
Hi,
I want to know how long you spent in training?
Hi,
Thank you for making your work available. In the paper it was stated that "Convergence is reached at about 300k iterations." under "Implementation details" in section "3.4. Experiments". What is the criteria you used to determine the convergence? Thanks once again.
Amazing work!
Would you like to provide the visualization code or pseudo algorithm on stereo matching?
Could you please include the requirments.txt?
It had some problems with opencv and I'm not sure what are the versions of the libraries you used.
It's also appreciated if you could include python 3 version. 2to3 didn't work for me, and I'm not sure it was due to the requirements version or not.
I noticed that in file loss.py, you say ' # empirically we find timing 0.005 tend to better performance'.
Do the results in the paper adopt this strategy? Do the file SpixelNet_bsd_ckpt.tar in pretrain_ckpt folder adopt this strategy?
I run the run_demo.py. Then,
RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a CPU-only machine, please use torch.load with map_location=torch.device('cpu') to map your storages to the CPU.
But my torch.cuda.is_acailable() is True.
What's wrong?
Hi,
My PC gpu is not strong enough to run 300k epoch for training, could you please send me your BSDS train data for 300k epochs?
email: [email protected]
thanks,
Dear authors,
The paper and the code are impressive. Specially, I am very impressive about the idea that modify PSMNet for stereo matching to simultaneously predict superpixels and disparities. But saddly, I can't find the code about the PSMNet. And I can't see the effect of stereo Matching which benifit form superpixel. If you have time, would you mind answer my question.
Thanks!
Hi, thanks for your great work and public code.
Can you provide all the results.csv files of other methods so I can draw the curves as in your paper?
Thanks!
Hi all,
你们的工作非常有趣!
但是请问这个模型可以生成指定数量的superpixel吗?还是只能生成固定数量的呢?
多谢多谢!
The input size must be same as the patch size in training, or the performance will drop?
How can I train the model with my own dataset?
Hi!
In the experiments, u only use the semantic loss and didn't use the slic loss? what u mentioned in other issues 'color loss' is represented the slic loss in the essay?
when I use the custom dataset, the loss_sem were very large(eg:17812.12), is that correctly?
second, Is the code: rgb2lab_torch is universal? Or is for BSD dataset only.
Thank u very much!
Hello, I don't know how to write upfeat and poolfeat pixel formula, can you provide me the pixel formula for upfeat and poolfeat image using Q?
I check your code, maybe label2one_hot_torch have some bug? Because I don't see you to convert labels from N x H x W to N x 1 x H x W .
for i, (input, label) in enumerate(train_loader):
iteration = i + epoch * epoch_size
# ========== adjust lr if necessary ===============
if (iteration + 1) in args.milestones:
state_dict = optimizer.state_dict()
for param_group in state_dict['param_groups']:
param_group['lr'] = args.lr * ((0.5) ** (args.milestones.index(iteration + 1) + 1))
optimizer.load_state_dict(state_dict)
# ========== complete data loading ================
label_1hot = label2one_hot_torch(label.to(device), C=50)
def label2one_hot_torch(labels, C=14):
b, _, h, w = labels.shape
one_hot = torch.zeros(b, C, h, w, dtype=torch.long).cuda()
target = one_hot.scatter_(1, labels.type(torch.long).data, 1)
Hi! Thanks for your share.
When I use my own dataset, I met the problem : "local variable ' slic_loss' referenced before assignment", but when I use BSD500 to train I didn't meet this problem. Can u know this is why?
Thanks for your patiance!
Hy!
I'm student of research. I'm interested in your code of Superpixel_fcn and I like it strongly.
I want to use this code in own research era.
During installation of cuda 9 , occur errors " graphic driver not compatible with this version of windows ". How I solve it...
In your code which GPu have used for cuda 9.
I have some troubles to install cuda 9. Please help me for solve my problem....
Hello Fengting,
First of all I want to say thank you for for your work and the provided code. Working with it is great but I have the following issue: I'm trying to jointly learn superpixels and an optical flow net.
I use the sintel dataset, cielab color features and the color loss like you did in the stereo application #12 (comment). The problem I have is that I do not learn super pixels. To further narrow down the problem I freeze the optical flow net and start with the provided model (pretrained on the segmentation task). After a few iterations (around 300) I have unlearned the superpixels (they are squares). I experimented with the pos weight parameter but it does not have an effect on the superpixels.
Do you have any idea what the problem is?
When I change the Variable "downsize" to 32, it becomes quite hard for the model to training, that is, the loss reduces slowly (have already trained 800 epochs).
Therefore, can I chose my own superpixels? Is there ant special hyparameter that should be changed accordingly?
Dear authors,
Your work seems very simple and effective (compared to SSN) at grouping the pixels to superpixels and I am wondering whether you have tried it on the Cityscapes semantic segmentation track.
It would be great if you could share with me your conclusion or observation on the semantic segmentation tasks.
Besides, we are also wondering whether you could share the reproduced implementations of the other superpixel methods, e.g., SSN, to ensure the comparison under the fair settings.
Thanks!
Hi!
Thanks for open-sourcing this awesome work.
I have some questions about how to use to train the downstream task?
I noticed that you said in the essay that you develop a downsampling/upsampling scheme for deep networks with the goal of generating high-resolution outputs for dense prediction tasks. Can you share with me about the job? Or can you give me some ideas about how to do?
I'll appreciate it if you replied to me! Thanks!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.