xuebinqin / basnet Goto Github PK
View Code? Open in Web Editor NEWCode for CVPR 2019 paper. BASNet: Boundary-Aware Salient Object Detection
License: MIT License
Code for CVPR 2019 paper. BASNet: Boundary-Aware Salient Object Detection
License: MIT License
Hi @Nathanua , thanks for your great work. Recently I am working on Salient object detection and I was trying to re-train BASNet from scratch to benchmark our own work. However I found I cannot reproduce reported result. I got around 0.078 in MAE on DUTS-TE(Trained on DUTS-TR), and I used the evaluation code you provided here https://github.com/NathanUA/Binary-Segmentation-Evaluation-Tool
Could you please give me some hint or explanation? thanks in advance!
I wonder, how to train only the refinement module? Cause I don't get familiar with Pytorch. Thank you.
Hello @Nathanua thank you for the great work and code. I have several problems as follows.
pred
) and ground truth (gt
) with threshold = 0.5pred
and gt
, how to do it? what is your kernel size? is it 3x3 as stated in hereXOR
operation in eroded
and non-eroded
version which means the different pixels will be white and the same pixels will be black (it will be like edge only).predXOR
and gtXOR
however I am still confused about slack p
where it set to be 3 pixels. then the relaxed F measure is similar computation with the original F measure.-Thank you-
I trained my data set, with a total of 1600 pictures, and trained 9 epoch, 1153 iterations. Batchsize set 12, but the total loss was always around 14-15, and occasionally changed to 13.8-13.9. Is this normal?
您好,感谢您的文章。想请教一下SSIM损失中关注边缘的主要原因是因为patch还是因为损失本身呢?换句话说如果用其他损失如DiceLoss,也应用到patch级别会有帮助嘛,谢谢!
文中对比的方法有Picanet、PAGRN、DGRL、RAS等等,
可以提供一下,这些方法测试得到的所有测试集的图片吗?
谢谢您!
Hi, again, thanks for your work and paper!
During my search of the model, I found some of them do not have pre-calculated saliency map, like PiCANet, DSS(no DUTSTE), etc.
but in your paper, you include them for PR curve comparison.
Do you mind sharing those pre-calculated saliency maps as well? It would be great for other research people in future if they need to access them.
Really thanks if it is possible to share.
作者您好,想请教您一下,paper里面loss热图那里(图5)提到的前景和后景的预测概率指的是什么呀,我不是很理解,谢谢。
感谢您的工作!
我想请问一下在basnet_test.py文件的最后,对于输出结果进行了一个归一化操作normPRED
def normPRED(d):
ma = torch.max(d)
mi = torch.min(d)
dn = (d - mi) / (ma - mi)
return dn
通常分割将网络输出层sigmoid之后作为最终结果,为什么在这项工程中加入了这种归一化操作,是有什么其他的考虑吗? 期待着您的回复!
大佬666
Hi
Thank you for the great work on this repository. How do I change the confidence value of the prediction in basnet_test.py ?
Thanks ... :)
Should be "hd6 = self.relu6d_2(self.bn6d_2(self.conv6d_2(hx)))", right?
Thank you for your answer, but how should this be used? I didn’t find any specific application in your U_2 code
Hello @Nathanua , thank you so much for your works. Here I still confuse how to evaluate your model on PASCAL-S dataset. As we know the pascal dataset Ground-truth is not binarize yet. It has value not only 0 or 255. How to process it and evaluate your model?
-Thank you-
你的iou loss怎么import进来的?
Hi,
I'm trying to run the test code but i'm getting the following error:
File "C:/CP/DL_REPOSITORIES/BASNet/basnet_test.py", line 76, in
for i_test, data_test in enumerate(test_salobj_dataloader):
File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 345, in next
data = self._next_data()
File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 841, in _next_data
idx, data = self._get_data()
File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 808, in _get_data
success, data = self._try_get_data()
File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 774, in _try_get_data
raise RuntimeError('DataLoader worker (pid(s) {}) exited unexpectedly'.format(pids_str))
RuntimeError: DataLoader worker (pid(s) 16732) exited unexpectedly
Can you please help?
Am trying to use TensorRT to speed up the inference. I converted my model trained from scratch to ONNX to feed to TensorRT. But TensorRT is throwing this error while parsing the model,
While parsing node number 186 [Gather]:
ERROR: onnx2trt_utils.hpp:277 In function convert_axis:
[8] Assertion failed: axis >= 0 && axis < nbDims
[E] Unable to parse ONNX model file: ../data/int8_api/basnet.onnx
Any pointers on how to address this?
Thanks.
I use your pre-trained model on HKU-IS and test on the result which has 0.030 MAE and 0.99 max F_beta!!(save and load the saliency map) Do I use the wrong way to test? Could you share how you test?
我下载了您给出的预训练模型,并放在了所指定的文件夹内。
但是在运行test文件的时候,还是会从pytorch的网站上下载模型
i want to ask about this:
if ite_num % 2000 == 0: # save model every 2000 iterations
torch.save(net.state_dict(), model_dir + "basnet_bsi_itr_%d_train_%3f_tar_%3f.pth" % (ite_num, running_loss / ite_num4val, running_tar_loss / ite_num4val))
running_loss = 0.0
running_tar_loss = 0.0
net.train() # resume train
ite_num4val = 0
why running_loss , tar_loss and ite_num4val is set to 0 after saving the model ? thankss
我想请问一下论文图一中的 boundary 是怎么生成的?在原论文中好像没有相关的描述,谢谢您!
Dear Nathan, i've used your BASNet code but i'm confused, why test data and test result are in different orientation? for example, i put horizontal orientation test data & the result is vertical orientation.
thank you.
Hello, i am curious about why the dilation of conv6d_m/conv6d_2 is different from the other stages like conv5d_m/conv5d_2. Is it a trick?
您好,我训练自己数据集的时候发现在15.6K迭代额时候,d0和d1的输出为nan,导致l0和l1损失为nan
`[epoch: 308/100000, batch: 2456/ 4085, ite: 156707] train loss: 2.115248, tar: 0.097755
l0: 0.090264, l1: 0.090268, l2: 0.094700, l3: 0.108498, l4: 0.157684, l5: 0.269343, l6: 0.561054
[epoch: 308/100000, batch: 2464/ 4085, ite: 156708] train loss: 2.114669, tar: 0.097731
l0: 0.110660, l1: 0.110660, l2: 0.116909, l3: 0.147194, l4: 0.230913, l5: 0.414125, l6: 0.675684
[epoch: 308/100000, batch: 2472/ 4085, ite: 156709] train loss: 2.115880, tar: 0.097773
l0: 0.101519, l1: 0.101512, l2: 0.107206, l3: 0.128373, l4: 0.198813, l5: 0.377140, l6: 0.674387
[epoch: 308/100000, batch: 2480/ 4085, ite: 156710] train loss: 2.116687, tar: 0.097785
l0: 0.092943, l1: 0.092937, l2: 0.097863, l3: 0.117802, l4: 0.182888, l5: 0.299898, l6: 0.505494
[epoch: 308/100000, batch: 2488/ 4085, ite: 156711] train loss: 2.115991, tar: 0.097769
l0: 0.104595, l1: 0.104529, l2: 0.109673, l3: 0.131785, l4: 0.201885, l5: 0.407138, l6: 0.842563
[epoch: 308/100000, batch: 2496/ 4085, ite: 156712] train loss: 2.118025, tar: 0.097791
l0: nan, l1: nan, l2: 2.413359, l3: 2.419617, l4: 2.441422, l5: 2.419549, l6: 2.403301
[epoch: 308/100000, batch: 2504/ 4085, ite: 156713] train loss: nan, tar: nan
l0: nan, l1: nan, l2: 2.489194, l3: 2.498003, l4: 2.527905, l5: 2.497976, l6: 2.474765
`
I want to re -train the basnet.pth model with my custom data set . I use the following code in the bastnet_train.py
model_dir_path = './saved_models/basnet_bsi/basnet.pth'
net = BASNet(3, 1)
net.load_state_dict(torch.load(model_dir_path))
if torch.cuda.is_available():
net.cuda()
I add the following line to load the existing model weights.
net.load_state_dict(torch.load(model_dir_path))
After training with my custom data set the model performance become down compare to the before re-train model . But working good with the similar type of data by which i re-train with.
Please guide me.
Dear author, there are still some missed checks when testing this code? How should I pre-process the data set (e.g. brighten or darken?) to ensure that no inspection is missed (even a few more notable marks are fine)?
The result on segmentation looks pretty good, but as the result is only a map image, how could I detect the objects in it? for example, detect cats, dogs in it with bounding boxes, I have applied the findcontour function on the resulted map, there are too much contours on a single "area"
Hi Xuebin Qin,
Thank you for your great work. However, I have a problem. When I train your model with batch size =1, it has too low performance that is far from batch size=8. How to train with bach size=1 and still have good performance?
Thank you
有没有测试代码的repo
Hello,Thank you very much for your work ! Did you test the thur15 dataset?
尊敬的作者您好,
我认为您在论文中关于损失函数的分析相当的精彩。但是我有几个疑问:1)这些展示图是如何制作出来的?(个人猜测是保存不同训练阶段的模型,处理同一幅图片,然后可视化三个loss)
2)为什么SSIM对边缘的不一致很敏感?根据图片展示确实如此,有理论上的解释吗?(其实本质问题是我想知道您的hybrid loss是理论指导设计还是实验指导设计的。)
My machine is 1080ti, which is running short of gpu memory.
HI, you mentioned that SSIM loss is used to encourage the original image structure in your paper. And it's reasonable. Recently, I want to add ssim loss to image segmentation to increase the building extraction performance. In my experiment, the ssim training loss is convergence well while the SSIM result between pred segmentation result and GT is lower than before.
I'm really confused. Any answer or suggestion will be thankful!
尊敬的作者:谢谢你的回复,我在测试自己的视频的时候经常出现输入图像尺寸不匹配的问题,但是我在你的U^2NET的代码中并没有看到关于upsample functions函数使用问题,希望你能够更加详细的帮我解答一下,
Dear Author:
Thank you for sharing the code. If I want to use your code to train my own data, is there any requirement for the size of the input training image?
Hello, I want to ask some questions about iou loss and ssim loss.
When I saw the code basnet_train.py, I found that :
ssim_out = 1 - ssim_loss(pred,target)
iou_out = iou_loss(pred,target)
I think that iou loss still need to be subtracted by 1 like that:
iou_out = 1 - iou_loss(pred,target)
But why it do not need to do that?
Thanks you!
Dear Nathan, i've used your BASNet code but i'm confused.
based on my experiment, is it true test data and test result are in different orientation? for example, i put horizontal orientation test data & the result is vertical orientation. sorry if wrong.
can you explain what is running_loss, running_tar_loss, ite_num4val and tar ?
thank you before :)
Hi. This model of GPU memory is large. Have you tried to optimize it? Thanks
Hi,
Thanks for your code!
Recently I need to re-train this network, on another dataset, HKU-IS, the model converges good,
but I don't know when to stop it...
the loss now in your code is 0.099 and 2.83 respecitively...
Because we don't have evaluation during the training...thus I do not know how good is the result now...
can you tell us when to stop the training when you train the BASNet using DUT-TR dataset?
作者您好,我看了一些您的training代码,发现你在训练过程中并没有validation阶段,是不需要吗?那停止训练判断的标准是loss收敛或者上升吗?
其次,您在论文中的可视化结果,是直接模型输出(概率图)-》归一化[0,1]-》保存为RGB(0-255)后直接展示,还是说您在展示前使用了某个阈值(假如阈值是0.5)将模型输出图片(像素值为0到255范围的整数)变为mask(像素值只有0或者255)了?
代码没有条件跑,弱弱的问一下,labels_v从dataloader中取出好,没有做尺度变换,直接给d0~d7做loss了,这里是如何操作的?没有看明白。
Hello, because I am interesting in both loss and want to do some experiment.
Could both of them directly combined with semantic segmentation? Because this paper is for salient object detection.
Thanks You!
作者您好,很感谢您之前的回复,对我很有帮助。我根据您的readme下载了代码,预训练模型并在本地重新生成了结果,我对比了SOD数据集。本地生成的结果在指标上(MAE,maxF和S-measure)要稍微好于论文(即您所提供的结果),我想(如果我的猜想并不正确的话,请您指正)这可能是pytorch版本(1.1.0 vs 0.4.0)或者系统环境(我在linux服务器上重现结果,并在windows系统上使用sal_toolbox评估模型)。
但是,出现了一个问题,不论是您提供的结果,还是我所重现的结果,在PACSAL-S数据集上测试时,maxF均为NaN(AUC也是)。另注:PASCAL-S的gt为uint8格式(SOD为logical格式),但是即使将格式进行转换后,测试指标仍然为NaN;除了PASCAL-S以外的其他数据集上的指标均是正常的。
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.